JP6789189B2 - Aircraft and portable electronics - Google Patents

Description

This application relates to flying objects and portable electronic devices.

Conventionally, there has been an air vehicle that can fly unmanned by remote control or automatic control (see, for example, Patent Document 1).

Japanese Patent No. 6143311

Conventional air vehicles have not been able to cooperate with the functions of portable electronic devices.

The flying object according to one of the embodiments is an flying object including a portable electronic device and a flying device to which the portable electronic device is detachably connected. The flying object changes a predetermined function of the portable electronic device when the portable electronic device connected to the flying device satisfies a predetermined condition.

The flying object according to one of the embodiments is an flying object including a portable electronic device and a flying device to which the portable electronic device is detachably connected. When the portable electronic device detects that the flying object is flying by the flying device, the predetermined function of the portable electronic device is changed.

The flying object according to one of the embodiments is a flying object including a portable electronic device and a flying device detachably attached to the portable electronic device. In the flying object, the portable electronic device controls the flight power of the flying device according to a predetermined function of the portable electronic device.

The portable electronic device according to one of the embodiments is a portable electronic device connected to the flight device, and includes a communication unit that communicates with the flight device and a controller that executes a predetermined function. The controller is connected to the flight device and changes the predetermined function when a predetermined condition is satisfied.

The portable electronic device according to one of the embodiments is a portable electronic device connected to the flight device, and includes a communication unit that communicates with the flight device, and a controller. The controller transmits information for controlling the flight power of the flying object to the flying device via the communication unit according to a predetermined function of the portable electronic device.

FIG. 1 is a diagram showing an example of an external configuration of a flight device according to an embodiment. FIG. 2 is a diagram showing an example of a method of attaching a mobile terminal to the flight device according to the embodiment. FIG. 3 is a diagram showing an example of the appearance configuration of the mobile terminal according to the embodiment. FIG. 4 is a diagram showing an example of the appearance configuration of the mobile terminal according to the embodiment. FIG. 5 is a diagram showing an example of the appearance configuration of the mobile terminal according to the embodiment. FIG. 6 is a diagram showing an example of the functional configuration of the mobile terminal according to the embodiment. FIG. 7 is a diagram showing an example of the functional configuration of the flying object according to the embodiment. FIG. 8 is a diagram showing an example of a screen of a mobile terminal. FIG. 9 is a flowchart showing an example of a processing procedure related to screen display control by a mobile terminal. FIG. 10 is a diagram showing an example of an operation related to the charging function of the mobile terminal. FIG. 11 is a flowchart showing a processing procedure of an example of control regarding the remaining battery level by the mobile terminal. FIG. 12 is a flowchart showing a processing procedure of another example of control regarding the remaining battery level by the mobile terminal. FIG. 13 is a flowchart showing a processing procedure of an example of control related to the notification function by the mobile terminal. FIG. 14 is a flowchart showing a processing procedure of an example of control related to an input function by a mobile terminal. FIG. 15 is a diagram showing an example of switching the mail display function depending on the mobile terminal. FIG. 16 is a flowchart showing a processing procedure of an example of control regarding the mail display function by the mobile terminal. FIG. 17 is a diagram showing an example of switching the map display function depending on the mobile terminal. FIG. 18 is a flowchart showing a processing procedure of an example of control regarding the map display function by the mobile terminal. FIG. 19 is a diagram showing another example of switching the map display function depending on the mobile terminal. FIG. 20 is a flowchart showing a processing procedure of another example of control regarding the map display function by the mobile terminal. FIG. 21 is a flowchart showing a processing procedure of an example of control regarding the navigation function by the mobile terminal. FIG. 22 is a flowchart showing a processing procedure of an example of control of a health function by a mobile terminal in flight.

A plurality of embodiments according to the present application will be described in detail with reference to the drawings. In the following description, similar components may be designated by the same reference numerals. Further, duplicate description may be omitted. In addition, matters that are not closely related to the description of the embodiment of the present application may be omitted from the description and illustration.

FIG. 1 is a diagram showing an example of an external configuration of a flight device according to an embodiment. FIG. 1 includes a plan view V1 and a front view V2 of the flight device 100 according to the embodiment. In the following description, the flight device 100 flies by the lift and propulsion force generated by the rotary blades driven by a motor or other power mechanism.

As shown in FIG. 1, the flight device 100 includes a main body 110, a connecting frame 130, rotary wings 150a to 150d, legs 170a and 170b, and a camera 190. The flight device 100 is configured by a plurality of connecting frames 130 connected to each other between the main body 110 and the rotor blades 150a to 150d. The main body 110 has a terminal mounting unit 111. When the flight device 100 is not in flight, the flight device 100 stands by with the legs 170a and 170b grounded.

The external configuration of the flight device 100 shown in FIG. 1 is an example, and the external configuration formed by the main body 110, the connecting frame 130, the rotary wings 150a to 150d, and the legs 170a and 170b, and the components such as the connecting frame and the rotary wings. The number of the above is not necessarily limited to the example shown in FIG.

FIG. 2 is a diagram showing an example of a method of attaching a mobile terminal to the flight device according to the embodiment. As shown in FIG. 2, for example, the mobile terminal 1 is detachably attached to the terminal mounting unit 111 so that the front surface of the mobile terminal 1 faces the plane side (z-axis plus direction) of the flight device 100. When the mobile terminal 1 is mounted on the terminal mounting unit 111, a part of the housing of the mobile terminal 1 (hereinafter referred to as "housing") (a part of the side face 1C described below) is a terminal mounting unit. It comes into contact with the protrusions 111a and 111b of the 111 without any gap. The protrusions 111a and 111b may be made of an elastic material having appropriate elasticity.

As shown in FIG. 2, the flying object 100F is the one in which the mobile terminal 1 is attached to the flight device 100. The flying object 100F includes a mobile terminal 1 and a flight device 100 to which the mobile terminal 1 is detachably connected. The detachable connection of the mobile terminal 1 to the flight device 100 includes, for example, the detachable attachment of the mobile terminal 1 to the flight device 100. The detachable connection of the mobile terminal 1 to the flight device 100 includes, for example, that the mobile terminal 1 is ready to fly together with the flight device 100. The detachable connection of the mobile terminal 1 to the flight device 100 includes, for example, that the mobile terminal 1 is attached to the flight device 100 so that the flight power of the flight device 100 can be controlled. In the following description, the flight device 100 to which the mobile terminal 1 is not attached may be referred to as a flying object 100F.

The front surface of the mobile terminal 1 is a surface that faces or contacts the user who uses the mobile terminal 1, and may be described as "front surface" or "display surface" in the following description. In the following description, the surface opposite to the "front" may be referred to as the "back". The mobile terminal 1 is an example of a "portable electronic device".

3 to 5 are views showing an example of the appearance configuration of the mobile terminal according to the embodiment. As shown in FIGS. 3 to 5, the mobile terminal 1 has a housing 1H. The surface forming the surface of the housing 1H includes a front face 1A, a back face 1B corresponding to the back surface of the front face 1A, and side faces 1C1 to 1C4 connecting the front face 1A and the back face 1B. In the following, the side faces 1C1 to 1C4 may be collectively referred to as the side faces 1C without specifying which surface the side faces 1C1 to 1C4 are.

The mobile terminal 1 has a touch screen 2B, buttons 3A to 3C, an illuminance sensor 4, a proximity sensor 5, a receiver 7, a microphone 8, and a camera 12 on the front face 1A. The mobile terminal 1 has a speaker 11 and a camera 13 on the back face 1B. The mobile terminal 1 has buttons 3D to 3F and a connector 14 on the side face 1C. In the following description, the buttons 3A to 3F may be collectively referred to as the button 3 without specifying which button it is.

The mobile terminal 1 has a pressure sensor 19 arranged along the side face 1C3 and the side face 1C4. The pressure sensor 19 can detect the pressure on the side face 1C3 and the side face 1C4. The pressure sensor 19 can sense, for example, the pressure exerted by the protrusions 111a and 111b on the side faces 1C3 and 1C4 when the mobile terminal 1 is mounted on the terminal mounting unit 111 of the flight device 100.

FIG. 6 is a diagram showing an example of the functional configuration of the mobile terminal according to the embodiment. As shown in FIG. 6, the mobile terminal 1 includes a touch screen display 2, a button 3, an illuminance sensor 4, a proximity sensor 5, a communication unit 6, a receiver 7, a microphone 8, a storage 9, and a controller. 10, speaker 11, camera (in-camera) 12, camera (out-camera) 13, connector 14, acceleration sensor 15, orientation sensor 16, angular velocity sensor 17, pressure sensor 18, pressure sensor 19 The GPS receiver 20, the battery 21, the charging unit 22, and the projector 23 are included.

The touch screen display 2 has a display 2A and a touch screen 2B. The display 2A and the touch screen 2B may be located, for example, overlapping, side by side, or apart. When the display 2A and the touch screen 2B are positioned so as to overlap each other, for example, one or more sides of the display 2A may not be along any side of the touch screen 2B.

The display 2A includes a liquid crystal display (LCD: Liquid Crystal Display), an organic EL display (OELD: Organic Electro-Luminence Display), an inorganic EL display (IELD: Organic Electro-Luminence Display), and the like. The display 2A displays objects such as characters, images, symbols, and figures on the screen. The screen including the objects displayed by the display 2A includes a screen called a lock screen, a screen called a home screen, and an application screen displayed while the application is being executed. The home screen may be referred to as a desktop, a standby screen, an idle screen, a standard screen, an application list screen, or a launcher screen.

The touch screen 2B detects contact or proximity of a finger, pen, stylus pen, or the like to the touch screen 2B. The touch screen 2B can detect a position on the touch screen 2B when a plurality of fingers, pens, stylus pens, or the like touch or approach the touch screen 2B. In the following description, a position where a plurality of fingers, pens, stylus pens, etc. detected by the touch screen 2B are in contact with or close to the touch screen 2B is referred to as a “detection position”. The touch screen 2B notifies the controller 10 of the contact or proximity of the finger to the touch screen 2B together with the detection position. The touch screen 2B may notify the controller 10 of the detection of contact or proximity with the notification of the detection position. The touch screen display 2 having the touch screen 2B can execute the operation that the touch screen 2B can perform. In other words, the operation performed by the touch screen 2B may be performed by the touch screen display 2.

The controller 10 has at least one of the contact or proximity detected by the touch screen 2B, the detection position, the change in the detection position, the duration of the contact or proximity, the interval at which the contact or proximity is detected, and the number of times the contact is detected. The type of gesture is determined based on the above. The mobile terminal 1 having the controller 10 can execute the operation that the controller 10 can perform. In other words, the operation performed by the controller 10 may be performed by the mobile terminal 1. Gesture is an operation performed on the touch screen 2B using a finger. The operation performed on the touch screen 2B may be performed by the touch screen display 2 having the touch screen 2B. Gestures that the controller 10 determines via the touch screen 2B include, for example, touch, long touch, release, swipe, tap, double tap, long tap, drag, flick, pinch in, and pinch out. Not limited to.

The detection method of the touch screen 2B may be any method such as a capacitance method, a resistance film method, a surface acoustic wave method, an infrared method, and a load detection method.

The button 3 receives an operation input from the user. The button 3 has, for example, buttons 3A to 3F. The number of buttons 3 may be any number. The controller 10 detects an operation on the button 3 by cooperating with the button 3. Operations on button 3 include, but are not limited to, click, double click, triple click, push, and multi-push, for example.

Buttons 3A to 3C are, for example, a home button, a back button, or a menu button. The button 3D is, for example, a power on / off button (power button) of the mobile terminal 1. Button 3D may also serve as a sleep / wake button. Buttons 3E and 3F are, for example, volume buttons.

The illuminance sensor 4 detects the illuminance. The illuminance is a value of a luminous flux incident on a unit area of the measurement surface of the illuminance sensor 4. The illuminance sensor 4 is used, for example, for adjusting the brightness of the display 2A.

The proximity sensor 5 detects the presence of a nearby object in a non-contact manner. The proximity sensor 5 has a light emitting element that irradiates infrared rays and a light receiving element that receives reflected light of infrared rays emitted from the light emitting element. The illuminance sensor 4 and the proximity sensor 5 may be configured as one sensor.

The communication unit 6 communicates wirelessly. The wireless communication standards supported by the communication unit 6 include, for example, cellular phone communication standards such as 2G, 3G, 4G, and 5G, and short-range wireless communication standards. Communication standards for cellular phones include, for example, LTE (Long Term Evolution), W-CDMA (registered trademark) (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (registered trademark), and GSM (registered trademark). Mobile communications), PHS (Personal Handy-phone System), and the like. Examples of short-range wireless communication standards include WiMAX (registered trademark) (Worldwide interoperability for Microwave Access), IEEE802.11, Bluetooth (registered trademark), Infrared Data Association (registered trademark), IrDA (Infrared Data Association), and (Infrared Data Association). ), WPAN (Wireless Personal Area Network) and the like. The communication unit 6 may support one or more of the above-mentioned communication standards.

The receiver 7 outputs a sound signal transmitted from the controller 10 as sound. The microphone 8 converts the input user's voice or the like into a sound signal and transmits it to the controller 10.

The storage 9 stores programs and data. The storage 9 may be used as a work area for temporarily storing the processing result of the controller 10. The storage 9 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage 9 may include a plurality of types of storage media. The storage 9 may include a combination of a storage medium such as a memory card, an optical disk, or a magneto-optical disk, and a reader of the storage medium. The storage 9 may include a storage device used as a temporary storage area such as a RAM (Random Access Memory).

The program stored in the storage 9 includes an application executed in the foreground or the background, and a support program (not shown) that supports the operation of the application. When the application is executed in the foreground, for example, the screen related to the application is displayed on the display 2A. The support program includes, for example, an OS (Operating System). The program may be installed in the storage 9 via wireless communication by the communication unit 6 or a non-transient storage medium.

The storage 9 includes a control program 9A, an aircraft cooperation program 9B, user search data 9C, image authentication data 9D, voice authentication data 9E, mail application 9F, navigation application 9G, calculation application 9H, and setting data 9Z. Can be memorized.

The control program 9A can each provide a function for realizing processing related to various operations of the mobile terminal 1. The function provided by the control program 9A includes a function of adjusting the brightness of the display 2A based on the detection result of the illuminance sensor 4. The function provided by the control program 9A includes a function of invalidating the operation on the touch screen 2B based on the detection result of the proximity sensor 5. The function provided by the control program 9A includes a function of realizing a telephone call by controlling the communication unit 6, the receiver 7, the microphone 8, and the like. The function provided by the control program 9A includes a function of controlling the shooting process of the camera 12 and the camera 13. The function provided by the control program 9A includes a function of controlling communication with an external device connected via the connector 14. The function provided by the control program 9A includes a function of performing various controls such as changing the information displayed on the display 2A according to the gesture determined based on the detection result of the touch screen 2B. The function provided by the control program 9A includes a function of detecting the movement, stop, etc. of the user carrying the mobile terminal 1 based on the detection result of the acceleration sensor 15. The function provided by the control program 9A includes a function of executing processing based on the current position based on the signal acquired from the GPS receiver 20.

The control program 9A can provide a function of determining whether or not the mobile terminal 1 (hereinafter, appropriately referred to as “own aircraft”) is attached to the flight device 100. The control program 9A can determine, for example, whether or not the aircraft is mounted on the flight device 100 based on the detection result of the pressure sensor 19. In the control program 9A, for example, the range (distribution) of the pressure acting on the side face 1C is the contact area between the protrusion 111a and the side face 1C3 of the terminal mounting unit 111, and the contact area between the protrusion 111b and the side face 1C3. If it is substantially the same as, the determination result that the own aircraft is attached to the flight device 100 may be derived.

The control program 9A can provide a function for pairing in a state in which communication with the flight device 100 is possible. When the control program 9A can confirm that the aircraft is attached to the flight device 100, the control program 9A sends an instruction for pairing the flight device 100 with the aircraft to the flight object cooperation program 9B.

The control program 9A can provide a function for determining whether or not the flight device 100 equipped with the own aircraft is in flight based on the detection result of the barometric pressure sensor 18. The control program 9A can provide a function for executing the zero point adjustment of the barometric pressure sensor 18 when the mobile terminal 1 is attached to the flight device 100.

The control program 9A can provide a function for starting a normal call on the condition that a notification indicating that a predetermined operation for the flight device 100 has been detected is received from the flight device 100 in response to an incoming call. .. The normal call corresponds to a call in which the transmitted voice is input to the microphone 8 and the received voice is output from the receiver 7.

The control program 9A can provide a function of executing image recognition with reference to the image authentication data 9D. The control program 9A executes image recognition of image data received from the flight device 100 or image data acquired by the mobile terminal 1, for example, and a gesture performed by the user of the mobile terminal 1 based on the result of the image recognition. Can recognize whether is a predetermined gesture. The predetermined gesture corresponds to a hand sign or the like performed by the user of the mobile terminal 1 toward the flight device 100 as a manifestation of intention to answer an incoming call, and each of the above gestures executed on the touch screen display 2. Is different.

The control program 9A can provide a function of executing voice recognition with reference to the voice authentication data 9E. In the control program 9A, for example, the voice data received from the flight device 100 or the own machine executes voice recognition of the voice data, and based on the result of the voice recognition, the voice input by the user of the mobile terminal 1 is predetermined. Can recognize whether it is voice. The predetermined voice corresponds to a line that the user of the mobile terminal 1 utters toward the flight device 100 as a manifestation of intention to answer the incoming call.

The control program 9A can provide a function of starting a speaker call when the predetermined gesture is recognized or the predetermined voice is recognized in response to an incoming call. The speaker call corresponds to a call in which the transmitted voice is input to the microphone 8 and the received voice is output from the speaker 11.

By linking with the control program 9A, the flight object cooperation program 9B can provide a function for realizing various processes in cooperation with the flight device 100. The flight object cooperation program 9B can, for example, transmit various commands to the flight device 100 to the flight device 100 after being translated into a code that can be executed by the flight device 100 itself.

The aircraft cooperation program 9B, for example, receives an instruction from the control program 9A and establishes a short-range wireless connection with the flight device 100 by using, for example, Bluetooth (registered trademark). It can be paired with the flight device 100. The flight object cooperation program 9B can transmit and receive various data to and from the flight device 100 by, for example, short-range wireless communication established with the flight device 100. For example, when an incoming telephone call is detected, the flight object cooperation program 9B can transmit a command instructing the user to approach the user within a predetermined distance range and data of an identifier included in the user search data 9C to the flight device 100. .. The aircraft cooperation program 9B can, for example, transmit a return instruction to the flight apparatus 100 when the end of the call is detected.

The user search data 9C is referred to when authenticating the user of the own machine. The user search data 9C includes, for example, data of an identifier uniquely assigned to a wearable terminal worn by a user of the own machine.

The image authentication data 9D is reference data for recognizing a predetermined gesture executed by the user of the own machine as a manifestation of intention to answer an incoming call. The image authentication data 9D is, for example, a template for recognizing a predetermined hand sign by pattern matching, a database previously constructed by machine learning using a multi-layer neural network for the purpose of recognizing a predetermined hand sign, or the like. including.

The voice authentication data 9E is reference data for recognizing a predetermined voice input by the user of the own machine as a manifestation of intention to answer an incoming call. The voice authentication data 9E is, for example, the voice information of the user of the mobile terminal 1. The voice information may be one that speaks any line, or may be one that speaks a specific line.

The mail application 9F provides a mail function for creating, sending, receiving, and displaying mail. The navigation application 9G provides a navigation function for directions and the like. The calculation application 9H can provide a function of calculating the number of steps, consumption, etc. of the user.

The setting data 9Z includes information on various settings related to the operation of the mobile terminal 1. The setting data 9Z includes, for example, a command (for example, answering an incoming call) to be executed when a predetermined gesture is recognized or a predetermined voice is recognized.

The mobile terminal 1 may cooperate with the cloud storage via the communication unit 6 to access the files and data stored in the cloud storage. The cloud storage may store a part or all of the programs and data stored in the storage 9.

The controller 10 includes an arithmetic processing unit. The arithmetic processing unit includes, for example, a CPU (Central Processing Unit), a System (System-on-a-Chip), an MCU (MicroControl Unit), an FPGA (Field-Programmable Gate Array), and a coprocessor. Not limited. The controller 10 comprehensively controls the operation of the mobile terminal 1 to realize various functions.

Specifically, the controller 10 executes an instruction included in the program stored in the storage 9 while referring to the data stored in the storage 9 as necessary. Then, the controller 10 controls the functional unit according to the data and the instruction, thereby realizing various functions. The functional unit includes, but is not limited to, for example, a display 2A, a communication unit 6, a microphone 8, a speaker 11, and a GPS receiver 20. The controller 10 may change the control according to the detection result of the detection unit. The detection unit includes, for example, a touch screen 2B, a button 3, an illuminance sensor 4, a proximity sensor 5, a microphone 8, a camera 12, a camera 13, an acceleration sensor 15, an orientation sensor 16, an angular velocity sensor 17, a pressure sensor 18, and a pressure sensor 19. Including, but not limited to.

By executing the control program 9A, the controller 10 can realize various controls related to the operation of its own machine. The controller 10 can realize, for example, a process of adjusting the brightness of the display 2A based on the detection result of the illuminance sensor 4. The controller 10 can realize, for example, a process of invalidating the operation on the touch screen 2B based on the detection result of the proximity sensor 5. The controller 10 can realize a process of realizing a telephone call by controlling, for example, the communication unit 6, the receiver 7, the microphone 8, and the like. The controller 10 can realize, for example, a process of controlling the shooting process of the camera 12 and the camera 13. The controller 10 can realize, for example, a process of controlling communication with an external device connected via the connector 14. The controller 10 can realize a process of performing various controls such as changing the information displayed on the display 2A according to the gesture determined based on the detection result of the touch screen 2B, for example. The controller 10 can realize, for example, a process of detecting the movement, stop, etc. of the user carrying the own machine based on the detection result of the acceleration sensor 15. The controller 10 can realize processing based on the current position based on, for example, a signal acquired from the GPS receiver 20.

By executing the control program 9A, the controller 10 can realize a process of determining whether or not its own aircraft is attached to the flight device 100 based on the detection result of the pressure sensor 19.

By executing the control program 9A and the flight object cooperation program 9B, the controller 10 can realize a process of pairing in a state in which communication with the flight device 100 is possible.

By executing the control program 9A and the flight object cooperation program 9B, the controller 10 issues an instruction to instruct the user to approach the user to a certain distance in response to the occurrence of an incoming call and an identifier included in the user search data 9C. The process of transmitting to the flight device 100 can be realized.

By executing the control program 9A, the controller 10 can realize an image recognition process for determining whether or not a predetermined gesture instructing the user to answer an incoming call has been performed.

By executing the control program 9A, the controller 10 can realize a voice recognition process for determining whether or not a voice input instructing the user to answer an incoming call has been performed.

The speaker 11 outputs a sound signal transmitted from the controller 10 as sound. The speaker 11 is used, for example, to output a ringtone and music. One of the receiver 7 and the speaker 11 may also serve as the other function.

The camera 12 and the camera 13 convert the captured image into an electric signal. The camera 12 is an in-camera that captures an object facing the display 2A. The camera 13 is an out-camera that captures an object facing the opposite surface of the display 2A. The camera 12 and the camera 13 may be mounted on the mobile terminal 1 in a functionally and physically integrated state as a camera unit that can be used by switching between an in-camera and an out-camera.

The connector 14 is a terminal to which another device is connected. The connector 14 includes USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), MHL (Mobile High-definition Link), Light Peak (Light Peak), Thunderbolt (Light Peak), Thunderbolt (Light Peak), Thunderbolt. It may be a general-purpose terminal such as a LAN connector (Local Area Network connector) or an earphone / microphone connector. The connector 14 may be a specially designed terminal such as a Dock connector. Devices connected to the connector 14 include, but are not limited to, for example, flying objects, chargers, external storage, speakers, communication devices, and information processing devices.

The acceleration sensor 15 can detect the direction and magnitude of the acceleration acting on the mobile terminal 1. As one example of the embodiment, a three-axis type acceleration sensor 15 that detects acceleration in the X-axis direction, the Y-axis direction, and the Z-axis direction can be adopted. The accelerometer 15 is a piezoresistive type, a capacitance type, a piezoelectric element type (piezoelectric type), a MEMS (Micro Electro Mechanical Systems) type based on a heat detection type, and a servo type that restores an operated movable coil by a feedback current. Alternatively, it can be configured by a strain gauge type or the like. The acceleration sensor 15 sends the detection result to the controller 10. The controller 10 can execute various controls based on the detection result of the acceleration sensor 15. For example, when the gravity acting on the mobile terminal 1 is output from the acceleration sensor 15 as an acceleration, the controller 10 can execute the control reflecting the direction of gravity acting on the mobile terminal 1.

The azimuth sensor 16 can detect the direction of the geomagnetism. The directional sensor 16 sends the detection result to the controller 10. The controller 10 can execute various controls based on the detection result of the directional sensor 16. For example, the controller 10 can specify the direction (direction) of the mobile terminal 1 from the direction of the geomagnetism and execute control reflecting the direction of the specified mobile terminal 1.

The angular velocity sensor 17 can detect the angular velocity of the mobile terminal 1. The angular velocity sensor 17 sends the detection result to the controller 10. The controller 10 can execute various controls based on the detection result of the angular velocity sensor 17. For example, the controller 10 can realize control that reflects the rotation of the mobile terminal 1 based on the presence or absence of the angular velocity output from the angular velocity sensor 17.

The controller 10 is not limited to the case where the detection results of the acceleration sensor 15, the direction sensor 16, and the angular velocity sensor 17 are used individually, and each detection result can be used in combination.

The atmospheric pressure sensor 18 can detect the atmospheric pressure acting on the mobile terminal 1. The detection result of the barometric pressure sensor 18 may include the amount of change in barometric pressure per unit time. The amount of change in atmospheric pressure may be an absolute value or a value obtained by accumulating a scalar amount. Any time may be set as the unit time. The barometric pressure sensor 18 sends the detection result to the controller 10.

The pressure sensor 19 can detect the pressure acting on the mobile terminal 1. The pressure sensor 19 may include a plurality of pressure sensitive elements. The pressure sensor 19 can also acquire information for the controller 10 to identify the range (for example, distribution) of the pressure acting on the mobile terminal 1 based on the detection result of the pressure sensor 19.

The GPS receiver 20 can receive radio signals in a predetermined frequency band from GPS satellites. The GPS receiver 20 performs demodulation processing of the received radio wave signal and sends the processed signal to the controller 10.

The battery 21 includes a rechargeable battery. The battery 21 includes, for example, a battery compatible with Qi (international standard for wireless power supply). The battery 21 supplies the stored electric power to each part or the like that requires electric power in the controller 10 and the mobile terminal 1. The controller 10, each part, and the like are operated by the electric power supplied from the battery 21.

The charging unit 22 controls the charging operation of the battery 21. The charging unit 22 receives power from a Qi-compatible charger in a non-contact manner using a charging coil, and can charge the battery 21 with the power. The charging unit 22 can charge the battery 21 from an external power source via the connector 14.

The projector 23 includes an image projection mechanism for projecting an image. The projector 23 has a light emitting unit that projects an image. The mobile terminal 1 can project an image on a wall surface, a screen, a road, or the like by projecting an image from a light emitting unit of a projector, that is, emitting light that constitutes the image. The operation of the projector 23 is controlled by the controller 10, and various images, videos, and the like sent from the controller 10 are projected and displayed.

The projector 23 is composed of a light source and an optical system that switches whether or not to project the light emitted from the light source according to the image data. For example, as the projector 23, a projector having a structure in which a halogen light, an LED light source, or an LD light source is used as a light source, and an LCD or a DMD (Digital Micro-mirror Device) is used as an optical system can be used. In this case, the optical system is arranged on the entire surface of the projection area corresponding to each pixel, and the image is projected on the entire surface of the projection area by turning the optical system on and off according to the light emitted from the light source. it can. Further, the projector 23 includes an optical system including a switching element that uses a laser beam as a light source and switches whether or not to transmit the light emitted from the light source, and a mirror that rasterly scans the light that has passed through the switching element. It is also possible to use a projector having the above configuration. In this case, the image can be projected on the projection area by changing the angle of the light emitted from the laser beam by the mirror and scanning the light emitted from the light source on the entire surface of the projection area.

The mobile terminal 1 may include a vibrator. The vibrator vibrates a part or the whole of the mobile terminal 1. The vibrator has, for example, a piezoelectric element, an eccentric motor, or the like in order to generate vibration. In addition to the above-mentioned sensors, the mobile terminal 1 may include a temperature sensor, a humidity sensor, and the like. The mobile terminal 1 implements a functional unit that is naturally used to maintain the function of the own device and a detection unit that is naturally used to realize control of the mobile terminal 1.

The mobile terminal 1 may access a storage server on the cloud via the communication unit 6 and acquire various programs and data.

A part or all of the program and data stored in the storage 9 in FIG. 6 may be downloaded from another device by wireless communication by the communication unit 6. A part or all of the programs and data stored in the storage 9 in FIG. 6 may be stored in a non-transient storage medium that can be read by the reading device included in the storage 9. A part or all of the program and data stored in the storage 9 in FIG. 6 may be stored in a non-transient storage medium that can be read by the reader connected to the connector 14. Non-transient storage media include, for example, optical disks such as CD (registered trademark), DVD (registered trademark), and Blu-ray (registered trademark), magneto-optical disks, magnetic storage media, memory cards, and solid-state storage media. Including, but not limited to.

The configuration of the mobile terminal 1 shown in FIG. 6 is an example, and may be appropriately changed as long as the gist of the present invention is not impaired. In the example shown in FIG. 6, the mobile terminal 1 includes two cameras, but the mobile terminal 1 may or may not have only one camera. In the example shown in FIG. 6, the mobile terminal 1 is provided with a plurality of sensors for detecting the position and the posture, but the mobile terminal 1 may not be provided with some of the sensors. Alternatively, the mobile terminal 1 may include other types of sensors for detecting at least one of position and orientation. The mobile terminal 1 shown in FIG. 6 includes a projector 23, but the projector 23 may not be provided. When the flight device 100 includes a projector, the mobile terminal 1 may project an image from the projector of the flight device 100.

FIG. 7 is a diagram showing an example of a functional configuration of the flight device according to the embodiment. As shown in FIG. 7, the main body 110 of the flight device 100 includes a communication unit 121, a connection unit 122, an imaging control unit 123, a power control unit 124, a sensor unit 125, a storage unit 126, and a control unit 127. And.

The communication unit 121 executes communication related to the exchange of various data with the mobile terminal 1. The communication unit 121 communicates with, for example, via a short-range wireless connection established with the mobile terminal 1.

The connection unit 122 is a terminal to which another device is connected. The connection portion 122 may be a general-purpose terminal such as USB.

The shooting control unit 123 controls shooting of an image using the camera 190. The control by the shooting control unit 123 includes the control of the shooting direction of the camera 190.

The power control unit 124 controls the driving force of the motors 140a to 140d. The motor 140a controls the rotation speed of the rotor blade 150a, the motor 140b controls the rotation speed of the rotary blade 150b, the motor 140c controls the rotation speed of the rotary blade 150c, and the motor 140d controls the rotation speed of the rotary blade 150d. To do.

The sensor unit 125 includes a plurality of sensors that detect data used for flight of the flight device 100 and control of equipment provided in the flight device 100. The sensor unit 125 includes a touch sensor 125a and a distance image sensor 125b. The touch sensor 125a includes any type of sensor such as a capacitance type, a resistive film type, a surface acoustic wave type, an ultrasonic type, an infrared type, an electromagnetic induction type, or a load detection type. The flight device 100 can detect that the flight device 100 is grasped by the user of the mobile terminal 1 by the touch sensor 125a. The touch sensor 125a may be located, for example, on the outer peripheral portion of the main body 110 of the flight device 100 or the connecting frame 130. Further, a handle for the user to grasp the flight device 100 may be provided on the flight device 100, and the touch sensor 125a may be arranged on the handle. The touch sensor 125a may be positioned so as to sandwich the installation portion of the touch sensor 125a. In order for the touch sensor 125a to be positioned so as to sandwich the installation portion, the sheet-shaped touch sensor 125a may be fixed so as to be wound around the installation portion, or at least two touch sensors 125a may be fixed to the installation portion. It may be arranged at a position sandwiching. The flight device 100 may detect that the touch sensor 125a has been grasped by the user when detecting the contact of the user at a position sandwiching the installation portion. The distance image sensor 125b can measure the distance to the object based on the time until the light such as the laser beam is reflected by the object and returns to the object. The distance image sensor 125b may radiate a laser beam to measure the direction of an object around the flight device 100 and the distance to the object.

The storage unit 126 can store programs and data. The storage unit 126 may include any non-transient storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage unit 126 may include a combination of a storage medium such as a memory card, an optical disk, or a magneto-optical disk, and a reader of the storage medium. The storage unit 126 may include a storage device used as a temporary storage area such as a RAM.

The storage unit 126 can store the control program 126a, the control data 126b, and the identifier data 126c. The control program 126a can each provide a function for realizing processing related to various operations of the flight device 100. The functions provided by the control program 126a include functions related to the control of the equipment provided in the flight device 100. The functions related to device control include a function for establishing a short-range wireless connection between the communication unit 121 and the mobile terminal 1 for pairing, a function for communicating with the mobile terminal 1 via the communication unit 121, and a shooting control unit. It includes a function for controlling the image capture of the camera 190 via 123. The function for communicating with the mobile terminal 1 includes notifying the mobile terminal 1 that a predetermined contact has been detected with the flight device 100.

The functions provided by the control program 126a include various functions related to flight control of the flight device 100. The function related to flight control includes a function for controlling the driving force of the motors 140a to 140d based on the detection result of the sensor unit 125. For example, the function for controlling the driving force of the motors 140a to 140d includes stopping the motors 140a to 140d when a predetermined operation on the flight device 100 is detected based on the detection result of the touch sensor 125a. .. The predetermined operation includes, for example, grasping at least one position of the connecting frame 130. The function related to flight control includes a function for adjusting the flight attitude of the flight device 100 based on the detection result of the sensor unit 125. The function related to flight control searches for the user of the mobile terminal 1 based on the identifier data 126c according to the instruction from the mobile terminal 1, and approaches the user of the mobile terminal 1 within a predetermined distance range based on the measurement result of the distance image sensor 125b. Including the function to do. The function related to flight control includes a function of returning to the standby station according to an instruction from the mobile terminal 1. The function related to flight control includes a function of moving to a charger according to an instruction from the mobile terminal 1.

The function related to flight control is a function of acquiring information indicating a positional relationship with an object around the flight device 100 based on the measurement result of the distance image sensor 125b, and a function of the flight device 100 based on the detection result of the sensor unit 125. It may include a function of detecting azimuth information regarding the angle (orientation) formed by the direction in which the nose faces, rotation angle information regarding the rotation angle centered on the vertical line passing through the center positions of the rotary blades 150a to 150d, and the like.

The control data 126b is referred to for performing processing related to various operations of the flight device 100.

The identifier data 126c is data of an identifier uniquely assigned to the wearable terminal by the user of the mobile terminal 1. The identifier data 126c corresponds to the identifier data included in the user search data 9C stored in the mobile terminal 1. The identifier data 126c is received from the mobile terminal 1 and stored in the storage unit 126.

The control unit 127 includes one or more arithmetic units. The arithmetic unit includes, for example, a CPU (Central Processing Unit), a System (System-on-a-Chip), an MCU (Micro Control Unit), an FPGA (Field-Programmable Gate Array), and a coprocessor, but is limited thereto. Not done. The control unit 127 realizes processing related to various operations of the flight device 100 by causing the arithmetic unit to execute the control program 126a. The control unit 127 may realize at least one part of the functions provided by the control program 126a by a dedicated IC (Integrated Circuit).

An example of the display function of the mobile terminal 1 will be described.
An example of changing the display function by the mobile terminal 1 will be described with reference to FIG. FIG. 8 is a diagram showing an example of screens 40A and 40B of the mobile terminal 1. The screens 40A and 40B include a home screen. The home screen may also be referred to as a desktop or standby screen. The screens 40A and 40B are displayed on the display 2A. The screens 40A and 40B are screens that allow the user to select which application to execute among the applications installed in the mobile terminal 1. The mobile terminal 1 executes the application selected on the screens 40A and 40B in the foreground. The application executed in the foreground causes the display 2A to display a screen different from the screens 40A and 40B.

The mobile terminal 1 can arrange icons on the screens 40A and 40B. A plurality of icons 50 are arranged on the screens 40A and 40B shown in FIG. Each icon 50 is associated with a specific application installed in the mobile terminal 1 in advance. When the mobile terminal 1 detects a specific gesture (for example, tap), voice request, or the like for the icon 50, the mobile terminal 1 executes the application associated with the icon 50. For example, the mobile terminal 1 executes the mail application 9F when a tap on the icon 50 associated with the mail application 9F is detected.

The icon 50 includes an image and a character string. The icon 50 may include a symbol or a figure instead of the image. The icon 50 does not have to include either an image or a character string. The icon 50 is arranged according to a predetermined rule. The wallpaper 41 is displayed behind the icon 50. The wallpaper 41 is sometimes called a photo screen or a back screen. The mobile terminal 1 can use any image as the wallpaper 41. The image is determined as the wallpaper 41 according to the user's settings, for example.

The screen 40A is displayed on the display 2A when the flight device 100 is not floating. The screen 40B is displayed on the display 2A when the flight device 100 is floating. On the screen 40B, a smaller number of icons 50 than the number of icons 50 on the screen 40A are arranged. The icon 50 on the screen 40B has a larger size than the icon 50 on the screen 40A.

For example, on the screen 40B, among the plurality of icons 50 on the screen 40A, the icon 50 having a high utilization rate may be preferentially arranged. For example, on the screen 40B, of the plurality of icons 50 on the screen 40A, only the icons 50 that can be used while the flight device 100 is floating may be arranged. The screen 40B does not have an icon 50 that cannot be used while the flight device 100 is floating. In the example shown in FIG. 8, the image and character string of the icon 50 on the screen 40B are larger than those of the icon 50 on the screen 40A.

An example of changing the display functions of the screens 40A and 40B by the mobile terminal 1 mounted on the flight device 100 will be described below.

In step S11 of FIG. 8, the mobile terminal 1 is attached to the flight device 100, and the flight device 100 is not floating. The mobile terminal 1 displays the screen 40A on the touch screen display 2. After that, the mobile terminal 1 instructs the flight device 100 to start the flight. In this case, the flight device 100 activates the motors 140a to 140d in response to the instruction from the mobile terminal 1 and starts floating. Floating of the flight device 100 includes, for example, ascent, flight, hovering, and the like of the flight device 100.

In step S12, the mobile terminal 1 causes the screen 40B to be displayed on the touch screen display 2 when the flight device 100 starts floating. As a result, the mobile terminal 1 changes the screen 40A displayed on the touch screen display 2 to the screen 40B corresponding to the floating of the flight device 100. The screen 40B is displayed on the touch screen display 2 of the mobile terminal 1 floating in the flight device 100.

For example, when the mobile terminal 1 detects a gesture, a hand sign, or the like for the user's icon 50 via the touch screen display 2, the mobile terminal 1 can execute an application corresponding to the icon 50. For example, the mobile terminal 1 can specify the icon 50 selected by the user based on the voice input via the microphone 8 and execute the application corresponding to the icon 50. For example, when the mobile terminal 1 detects a predetermined user gesture from an image taken by the camera, the mobile terminal 1 can execute the application of the icon 50 corresponding to the gesture, the hand sign, and the like. For example, the mobile terminal 1 can execute the application of the predetermined icon 50 when the flight device 100 detects the operation of grasping the connecting frame 130 by the user.

In the mobile terminal 1, even if the flight device 100 and the user are far apart, the size of the icon 50 on the screen 40B is larger than the icon 50 on the screen 40A, so that the visibility of the mobile terminal 1 is reduced by the user when floating. It can be suppressed and the operability can be improved.

In step S12, the mobile terminal 1 instructs the flight device 100 to end the flight. When the flight device 100 finishes floating, the mobile terminal 1 causes the screen 40A to be displayed on the touch screen display 2A as shown in step S11. The mobile terminal 1 can switch and display the screens 40A and 40B based on whether or not the flight device 100 is floating. As a result, the mobile terminal 1 does not require the user to switch between the screens 40A and 40B, so that the operability when mounted on the flight device 100 can be improved.

FIG. 9 is a flowchart showing an example of a processing procedure related to display control of screens 40A and 40B by the mobile terminal 1. The processing procedure shown in FIG. 9 is realized by the controller 10 executing the control program 9A. The processing procedure shown in FIG. 9 is repeatedly executed by the controller 10. The processing procedure shown in FIG. 9 is executed when the mobile terminal 1 is attached to the flight device 100 and the screen 40A is displayed on the display 2A.

As shown in FIG. 9, the controller 10 of the mobile terminal 1 determines whether or not the flight device 100 has started floating (step S101). For example, when the controller 10 instructs the flight device 100 to start floating, the controller 10 determines that the flight device 100 has started floating. For example, the controller 10 may determine the start of floating of the flight device 100 based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like. When the controller 10 determines that the flight device 100 has not started floating (No in step S101), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that the flight device 100 has started floating (Yes in step S101), the controller 10 proceeds to step S102.

The controller 10 changes the screen 40A displayed on the display 2A to the screen 40B when the flight device 100 is floating (step S102). When the screen 40B is displayed on the display 2A, the controller 10 advances the process to step S103.

The controller 10 determines whether or not the flight device 100 has finished floating (step S103). For example, when the controller 10 instructs the flight device 100 to end floating, the controller 10 determines that the flight device 100 has finished floating. For example, the controller 10 may determine the end of floating of the flight device 100 based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like. When the controller 10 determines that the flight device 100 has not finished floating (No in step S103), the flight device 100 continues to float, so the process returns to step S103 already described. When the controller 10 determines that the flight device 100 has finished floating (Yes in step S103), the controller 10 advances the process to step S104.

The controller 10 changes the screen 40B displayed on the display 2A to the screen 40A when the flight device 100 is not floating (step S104). When the screen 40B is displayed on the display 2A, the controller 10 ends the processing procedure shown in FIG.

An example of the charging function of the mobile terminal 1 will be described.
The control program 9A can provide a function of urging the user to charge the mobile terminal 1 when the remaining amount of the battery 21 becomes lower than the predetermined remaining amount. The control program 9A can provide a function of moving the mobile terminal 1 to the charger by the flight device 100 and automatically charging the battery 21 when the remaining amount of the battery 21 becomes lower than the predetermined remaining amount. The charger includes, for example, a non-contact charging stand, a non-contact charger and the like.

The setting data 9Z includes data indicating whether to enable or disable the function of automatically charging when the mobile terminal 1 is attached to the flight device 100. The setting data 9Z includes position information indicating the position where the charger is installed.

For example, the mobile terminal 1 executes a function of notifying the decrease of the remaining amount of the battery 21 when the remaining amount of the battery 21 becomes lower than the predetermined remaining amount and the own aircraft is not attached to the flight device 100. .. When the remaining amount of the battery 21 is lower than the predetermined remaining amount, the mobile terminal 1 changes the function of notifying the decrease of the remaining amount of the battery 21 when the own aircraft is attached to the flight device 100. For example, the mobile terminal 1 changes from the function of notifying the decrease in the remaining amount of the battery 21 to the function of automatically charging the battery 21. The function of automatically charging the battery 21 includes a function in which the mobile terminal 1 controls the flight power of the flight device 100 and causes the flight device 100 to move its own device to the charger.

FIG. 10 is a diagram showing an example of an operation related to the charging function of the mobile terminal 1. In step S21 of FIG. 10, the mobile terminal 1 detects that the remaining battery level is lower than the predetermined remaining amount in the state of being attached to the flight device 100. The mobile terminal 1 instructs the flight device 100 to ascend.

In step S22, the flight device 100 activates the motors 140a to 140d according to the instruction of the mobile terminal 1 and floats. The mobile terminal 1 searches for a charger 300 around the own machine by using the cameras 12 and 13 of the own machine, the camera 190 of the flight device 100, the GPS receiver 20 of the own machine, and the like. The search for the charger 300 includes, for example, a search using image recognition using an image of the charger 300, a search using preset position information of the charger 300, and the like. In the example shown in step S22, the mobile terminal 1 specifies the position of the charger 300. The mobile terminal 1 instructs the flight device 100 to fly based on the movement direction and the movement distance to the specified charger 300.

In step S23, the flight device 100 flies above the charger 300 and lands towards the charger 300. In the mobile terminal 1, the charging unit 22 is positioned near the charger 300 when the flight device 100 lands. In the mobile terminal 1, the charger 300 and the charging unit 22 charge the battery 21 in a non-contact manner.

When the remaining amount of the battery 21 is lower than the predetermined remaining amount while connected to the flight device 100, the mobile terminal 1 floats the flight device 100 and moves to the charger 300 to charge the battery 21. be able to. The state of being connected to the flight device 100 includes, for example, a state in which the mobile terminal 1 and the flight device 100 are paired, and a state in which the mobile terminal 1 and the flight device 100 are electrically connected by a cable or the like. As a result, the mobile terminal 1 can be improved in convenience because it can prevent the user from forgetting to charge the device when he / she uses his / her own device when he / she goes out.

FIG. 11 is a flowchart showing a processing procedure of an example of control regarding the remaining amount of the battery 21 by the mobile terminal 1. The processing procedure shown in FIG. 11 is realized by the controller 10 executing the control program 9A and the flight object cooperation program 9B. The processing procedure shown in FIG. 11 is repeatedly executed by the controller 10.

As shown in FIG. 11, the controller 10 of the mobile terminal 1 determines whether or not the remaining amount of the battery 21 is lower than the predetermined remaining amount (step S201). For example, the predetermined remaining amount includes a remaining amount for determining whether or not to perform a preset charging, a remaining amount set by a user or the like, and the like. When the controller 10 determines that the remaining amount of the battery 21 is not lower than the predetermined remaining amount (No in step S201), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that the remaining amount of the battery 21 is lower than the predetermined remaining amount (Yes in step S201), the controller 10 proceeds to the process in step S202.

The controller 10 determines whether or not the own aircraft is connected to the flight device 100 (step S202). When the own aircraft is connected to the flight device 100, for example, communication with the flight device 100 equipped with the own aircraft is established, and communication with the flight device 100 equipped with the own aircraft becomes possible. Including things like that.

When the controller 10 determines that the aircraft is not connected to the flight device 100 (No in step S202), the controller 10 proceeds to step S203. The controller 10 notifies that the remaining battery level is low (step S203). For example, the controller 10 causes the display 2A to display a notification screen for notifying the decrease in the remaining battery level. As a result, the mobile terminal 1 can notify the user of the decrease in the remaining battery level by displaying the notification screen. When the controller 10 gives a notification, the controller 10 ends the processing procedure shown in FIG.

When the controller 10 determines that the aircraft is connected to the flight device 100 (Yes in step S202), the controller 10 proceeds to step S204. The controller 10 transmits control information instructing the flight device 100 to ascend (step S204). For example, the controller 10 transmits control information including the ascending height and the like to the flight device 100 via the communication unit 6. As a result, the flight device 100 floats to a predetermined height with the mobile terminal 1 attached. After that, the controller 10 advances the process to step S205.

The controller 10 executes a specific process for identifying the position of the charger 300 based on the image taken by the camera (step S205). For example, the controller 10 that has executed the specific process takes a picture of the surroundings of the own aircraft or the flight apparatus 100 with the own aircraft or the camera of the flight apparatus 100 while rotating the flight apparatus 100 at a raised position. The controller 10 compares the captured image with the image of the charger 300 prepared in advance, and determines whether or not the charger 300 exists. When the charger 300 does not exist, the controller 10 stores in the storage 9 as a result that the position of the charger 300 could not be specified. When the charger 300 is present, the controller 10 calculates the position of the charger 300 based on information such as the current position of the own machine, the distance to the object measured by the proximity sensor 5, the distance sensor, and the like. When the specific process is completed, the controller 10 advances the process to step S206.

The controller 10 determines whether or not the position of the charger 300 can be specified (step S206). If the controller 10 determines that the position of the charger 300 cannot be specified (No in step S206), the process proceeds to step S207.

The controller 10 transmits control information instructing the flight device 100 to end the flight (step S207). For example, the controller 10 transmits control information instructing landing to the flight device 100 via the communication unit 6. As a result, the flight device 100 lands with the mobile terminal 1 attached. The controller 10 executes step S203 already described to notify that the remaining battery level is low. When the controller 10 gives a notification, the controller 10 ends the processing procedure shown in FIG.

When the controller 10 determines that the position of the charger 300 can be specified (Yes in step S206), the controller 10 proceeds to step S208. The controller 10 calculates the moving direction and the moving distance from the current position of the own machine to the charger 300 (step S208). The controller 10 stores the calculated moving direction, moving distance, and moving start position information in the storage 9 (step S209). The controller 10 transmits control information instructing flight based on the calculated movement direction and movement distance to the flight device 100 (step S210). As a result, the flight device 100 flies by the movement distance in the instructed movement direction with the mobile terminal 1 attached, and lands at the position toward the charger 300. When the controller 10 transmits the control information to the flight device 100, the process proceeds to step S211.

The controller 10 notifies the charging of the battery 21 (step S211). For example, the controller 10 causes the display 2A to display the charging screen of the battery 21. For example, the controller 10 can transmit information for activating the charger 300 via the communication unit 6 when the charger 300 is not activated. When the controller 10 notifies the charging of the battery 21, the processing procedure shown in FIG. 11 is terminated.

When the charging of the battery 21 is completed, the mobile terminal 1 may add a function of automatically returning from the charger 300 to the original place (the position where the flight is started) by the flight device 100. For example, when the mobile terminal 1 detects the end of charging of the battery 21, it calculates a return route based on information such as the movement direction, the movement distance, and the movement start position stored in the storage 9, and the flight of the return route. To the flight device 100.

When the remaining amount of the battery 21 approaches a predetermined remaining amount, the mobile terminal 1 may fly by the flight device 100 and perform automatic charging during a time period when the user does not use his / her own machine. In the time zone when the user does not use the own device, for example, the mobile terminal 1 may learn the behavior pattern of the user or let the user set it. The mobile terminal 1 can contribute to the reduction of the electricity charge by flying with the flight device 100 and performing automatic charging during the time when the electricity charge is low.

FIG. 12 is a flowchart showing a processing procedure of another example of control regarding the remaining amount of the battery 21 by the mobile terminal 1. The processing procedure shown in FIG. 12 is realized by the controller 10 executing the control program 9A and the flight object cooperation program 9B. The processing procedure shown in FIG. 12 is repeatedly executed by the controller 10.

In the example shown in FIG. 12, the processes of steps S201 to S203 and steps S209 to S211 are the same as the processes of steps S201 to S203 and steps S209 to S211 shown in FIG. 11, so only different parts will be described. However, the description of the same part will be omitted.

As shown in FIG. 12, the controller 10 of the mobile terminal 1 executes the processes of steps S201 to S203. When the controller 10 determines that the aircraft is connected to the flight device 100 (Yes in step S202), the controller 10 proceeds to step S221. The controller 10 determines whether or not the position information of the charger 300 is registered (step S221). For example, the controller 10 determines that the position information of the charger 300 is registered when the position information of the charger 300 is set in the setting data 9Z.

When the controller 10 determines that the position information of the charger 300 is not registered (No in step S221), the controller 10 proceeds to step S203 already described. The controller 10 executes step S203 already described to notify that the remaining battery level is low. When the controller 10 gives a notification, the controller 10 ends the processing procedure shown in FIG.

When the controller 10 determines that the position information of the charger 300 is registered (Yes in step S221), the controller 10 proceeds to the process in step S222. The controller 10 uses the GPS receiver 20 to acquire the current position of its own device (step S222). The controller 10 determines whether or not the charger 300 is within the movable range of the flight device 100 (step S223). For example, when the distance from the own device to the charger 300 is smaller than a predetermined distance, the controller 10 determines that the charger 300 is within the movable range of the flight device 100.

When the controller 10 determines that the charger 300 is not within the movable range of the flight device 100 (No in step S223), the controller 10 proceeds to step S203, which has already been described. The controller 10 executes step S203 already described to notify that the remaining battery level is low. When the controller 10 gives a notification, the controller 10 ends the processing procedure shown in FIG.

When the controller 10 determines that the charger 300 is within the movable range of the flight device 100 (Yes in step S223), the controller 10 proceeds to step S224. The controller 10 calculates the moving direction and moving distance from the current position of the own machine to the position of the charger 300 (step S224), and proceeds to step S209, which has already described the process.

The controller 10 stores the calculated moving direction, moving distance, and moving start position information in the storage 9 (step S209). The controller 10 transmits control information instructing flight based on the calculated movement direction and movement distance to the flight device 100 (step S210). As a result, the flight device 100 flies by the movement distance in the instructed movement direction with the mobile terminal 1 attached, and lands at the position toward the charger 300. When the controller 10 transmits the control information to the flight device 100, the process proceeds to step S211.

The controller 10 notifies the charging of the battery 21 (step S211). For example, the controller 10 can transmit information for activating the charger 300 via the communication unit 6 when the charger 300 is not activated. When the controller 10 notifies the charging of the battery 21, the processing procedure shown in FIG. 12 is terminated.

In the processing procedure shown in FIG. 12, when the controller 10 determines that the position information of the charger 300 is not registered (No in step S221), the case where the processing proceeds to step S203 already described will be described. Not limited to. For example, if the processing procedure shown in FIG. 12 is No in step S221, the processing procedure after step S204 in FIG. 11 may be executed to search for the charger 300.

An example of the notification function of the mobile terminal 1 will be described.
The control program 9A can provide a function of determining the moving state of the mobile terminal 1 (in other words, the moving state of the user carrying the mobile terminal 1). By executing the control program 9A, the mobile terminal 1 determines the moving state of the mobile terminal 1 based on the detection results and the determination data of the acceleration sensor 15, the orientation sensor 16, and the angular velocity sensor 17. For example, the plurality of moving states of the mobile terminal 1 include a stopped state, a stationary state, a walking state, a running state, a moving state in a vehicle, and a moving state in a bicycle. The stopped state includes a state in which a user carrying his / her own machine is stopped. The stationary state includes the state in which the aircraft is placed. The walking state includes a state in which a user carrying his / her own machine is walking. The running state includes a state in which a user carrying his / her own machine is running. The state of movement by vehicle includes a state in which a user carrying his / her own machine is moving by vehicle. Vehicles include, for example, automobiles, trains, buses, airplanes, motorcycles and the like. The state of traveling by bicycle includes a state in which a user carrying his / her own machine is traveling by bicycle.

For example, the mobile terminal 1 changes the notification function when a predetermined condition is satisfied while being connected to the flight device 100. The notification function includes, for example, a function of notifying an incoming call, an e-mail, an emergency information, or the like. The predetermined condition includes, for example, a condition that the moving state of the own aircraft is a moving state in a vehicle while being connected to the flight device 100. When the mobile terminal 1 is connected to the flight device 100 and satisfies a predetermined condition, the mobile terminal 1 gives a notification by the notification function in a state of being connected to the flight device 100 without floating the flight device 100. When the mobile terminal 1 is connected to the flight device 100 and does not satisfy a predetermined condition, the mobile terminal 1 floats the flight device 100 near the user and gives a notification by the notification function in the state of being floated by the flight device 100.

The mobile terminal 1 can diversify the notification to the user according to the change in the moving state of the own device. Since the mobile terminal 1 does not float the flight device 100 in the vehicle when the own aircraft connected to the flight device 100 is in a moving state by the vehicle, it is possible to improve the safety in the notification using the flight device 100. ..

FIG. 13 is a flowchart showing a processing procedure of an example of control related to the notification function by the mobile terminal 1. The processing procedure shown in FIG. 13 is realized by the controller 10 executing the control program 9A and the flight object cooperation program 9B. The processing procedure shown in FIG. 13 is repeatedly executed by the controller 10.

As shown in FIG. 13, the controller 10 of the mobile terminal 1 determines whether or not a predetermined event has occurred (step S301). Predetermined events include, for example, events such as receiving emails and notification information, receiving telephone calls, and the like. When the controller 10 determines that a predetermined event has not occurred (No in step S301), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that a predetermined event has occurred (Yes in step S301), the controller 10 advances the process to step S302.

The controller 10 determines the moving state of the own machine (step S302). For example, the controller 10 compares the acceleration pattern detected by the acceleration sensor 15 with the acceleration pattern for each state of the discrimination data, and discriminates the state corresponding to the acceleration pattern of the matching discrimination data as the state of the own machine. When the controller 10 determines the moving state of the own machine, the controller 10 advances the process to step S303.

The controller 10 determines whether or not the vehicle is in a moving state based on the determination result in step S302. When the controller 10 determines that the vehicle is in a moving state (Yes in step S303), the controller 10 proceeds to step S304. The controller 10 executes a process of notifying the occurrence of a predetermined event without suspending the flight device 100 (step S304). For example, the controller 10 causes the display 2A to display a notification screen indicating the occurrence of a predetermined event. For example, the controller 10 outputs a notification sound for notifying the occurrence of a predetermined event from the speaker 11. As a result, the mobile terminal 1 notifies the occurrence of a predetermined event without using the flight device 100. When the controller 10 notifies the occurrence of a predetermined event, the controller 10 ends the processing procedure shown in FIG.

When the controller 10 determines that the vehicle is not in the moving state (No in step S303), the controller 10 proceeds to step S305. The controller 10 transmits control information instructing the flight device 100 to ascend (step S305). For example, the controller 10 transmits control information including the ascending height and the like to the flight device 100 via the communication unit 6. As a result, the flight device 100 floats to a predetermined height with the mobile terminal 1 attached. After that, the controller 10 advances the process to step S306.

The controller 10 executes a specific process for identifying the position of the user based on the image taken by the camera (step S306). For example, the controller 10 that has executed the specific process takes a picture of the surroundings of the own aircraft or the flight apparatus 100 with the own aircraft or the camera of the flight apparatus 100 while rotating the flight apparatus 100 at a raised position. The controller 10 compares the captured image with the image of the user prepared in advance, and determines whether or not the user exists. When the user does not exist, the controller 10 stores in the storage 9 as a result that the position of the user could not be specified. When a user exists, the controller 10 calculates the position of the user based on information such as the current position of the own machine, the proximity sensor 5, the distance to the user measured by the distance sensor, and the like. When the specific process is completed, the controller 10 advances the process to step S307.

The controller 10 calculates the moving direction and the moving distance from the current position of the own machine to the user (step S307). The controller 10 transmits control information instructing flight based on the calculated movement direction and movement distance to the flight device 100 (step S308). As a result, the flight device 100 flies by the movement distance in the instructed movement direction with the mobile terminal 1 attached.

The controller 10 determines whether or not the flight device 100 has approached the user (step S309). For example, the controller 10 determines that the flight device 100 has approached the user when the distance to the user detected by the proximity sensor 5 or the like is smaller than a predetermined distance. When the controller 10 determines that the flight device 100 is not approaching the user (No in step S309), the controller 10 returns the process to step S306 already described. When the controller 10 determines that the flight device 100 is approaching the user (Yes in step S309), the controller 10 proceeds to step S310.

The controller 10 executes a process of notifying the occurrence of a predetermined event while the flight device 100 is suspended (step S310). For example, the controller 10 causes the display 2A to display a notification screen indicating the occurrence of a predetermined event. As a result, the mobile terminal 1 notifies the occurrence of the predetermined event in a state of being floated near the user by the flight device 100 in response to the occurrence of the predetermined event.

When the controller 10 notifies the occurrence of a predetermined event, the controller 10 instructs the flight device 100 to end the floating (step S311). For example, the controller 10 transmits control information instructing the end of levitation (for example, landing) to the flight device 100 via the communication unit 6. As a result, the flight device 100 lands with the mobile terminal 1 attached. After that, the controller 10 ends the processing procedure shown in FIG.

An example of the input function of the mobile terminal 1 will be described.
The control program 9A can provide a function of changing the character input function when connected to the flight device 100, depending on whether or not a predetermined condition is satisfied. The predetermined condition includes, for example, a condition that the flight device 100 is floating when connected to the flight device 100.

The setting data 9Z includes data indicating whether to enable or disable the function of changing the character input according to a predetermined condition when the mobile terminal 1 is attached to the flight device 100. The mobile terminal 1 can determine whether or not to enable the function of changing the character input according to a predetermined condition by setting the setting data 9Z.

For example, when the mobile terminal 1 is connected to the flight device 100 and the flight device 100 is not floating, the mobile terminal 1 displays an input screen using keys on the display 2A, and inputs input data according to an operation on the keys on the input screen. It is possible to provide an input function to accept. When the mobile terminal 1 is connected to the flight device 100, when the flight device 100 is floating, the mobile terminal 1 displays an input screen by voice input on the display 2A and accepts the voice data input by the microphone 8 as input data. Can provide input function.

FIG. 14 is a flowchart showing a processing procedure of an example of control related to the input function by the mobile terminal 1. The processing procedure shown in FIG. 14 is realized by the controller 10 executing the control program 9A. The processing procedure shown in FIG. 14 is repeatedly executed by the controller 10.

As shown in FIG. 14, the controller 10 of the mobile terminal 1 determines whether or not a character input event has occurred (step S401). The character input event includes, for example, a character input event that occurs when an application such as text, mail, telephone, or browser is executed. When the controller 10 determines that the character input event has not occurred (No in step S401), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that the character input event has occurred (Yes in step S401), the controller 10 proceeds to step S402.

The controller 10 determines whether or not the flight device 100 is floating (step S402). For example, when the controller 10 instructs the flight device 100 to float, the controller 10 determines that the flight device 100 is floating. For example, the controller 10 may determine whether or not the flight device 100 is floating based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like.

When the controller 10 determines that the flight device 100 is not floating (No in step S402), the controller 10 proceeds to step S403. The controller 10 executes an input process based on the input screen by the key (step S403). The input process based on the input screen by the key includes a process of displaying the input screen including the key on the display 2A and accepting the character corresponding to the key as input data when the input operation for the key is detected. The input process based on the key input screen is executed until, for example, an end operation or the like is detected. When the input is completed, the controller 10 erases the input screen from the display 2A (step S405). When the controller 10 erases the input screen, the controller 10 ends the processing procedure shown in FIG.

When the controller 10 determines that the flight device 100 is floating (Yes in step S402), the controller 10 proceeds to step S404. The controller 10 executes an input process based on the voice input screen (step S404). The input process based on the voice input screen includes a process of displaying an input screen prompting voice input on the display 2A and accepting characters or character strings corresponding to the voice input via the microphone 8 as input data. The input process based on the voice input screen is executed until, for example, the end voice or the like is detected. When the input by voice is completed, the controller 10 erases the input screen from the display 2A (step S405). When the controller 10 erases the input screen, the controller 10 ends the processing procedure shown in FIG. Further, when the controller 10 determines that the flight device 100 is floating, the controller 10 may perform only the input processing by voice without displaying the input screen. In this case, the controller 10 may output the input content of the input screen that is not displayed from the speaker 11 by voice to assist the user in performing a predetermined input by voice.

For example, when the mobile terminal 1 connected to the flight device 100 is floating, it may be difficult for the user to perform a gesture such as touching or flicking on the mobile terminal 1. When the mobile terminal 1 is connected to the flight device 100, when the flight device 100 is floating, by switching from key input to voice input, non-contact input becomes possible, and operability is reduced. It can be suppressed. Since the mobile terminal 1 can receive input in a floating state by the flight device 100, it is possible to provide the user with a novel effect.

An example of the mail display function of the mobile terminal 1 will be described.
The mail application 9F can provide a function of changing the function of displaying a mail when the own aircraft is connected to the flight device 100, depending on whether or not a predetermined condition is satisfied. The predetermined condition includes, for example, a condition that the flight device 100 is floating when the own aircraft is connected to the flight device 100.

The setting data 9Z includes data indicating whether to enable or disable the function of changing the display of the mail according to a predetermined condition when the mobile terminal 1 is attached to the flight device 100. The mobile terminal 1 can determine whether or not to change the mail display function according to a predetermined condition by setting the setting data 9Z.

FIG. 15 is a diagram showing an example of switching the mail display function by the mobile terminal 1. As shown in step S51 of FIG. 15, when the mobile terminal 1 is connected to the flight device 100, the mail screen 45A is displayed on the touch screen display 2 when the flight device 100 is not floating. The mail screen 45A displays the title, sender, and body of the mail in a predetermined character size. The predetermined character size includes, for example, a character size preset in the mobile terminal 1 and when the flight device 100 is not floating. As shown in step S52 of FIG. 15, when the mobile terminal 1 is connected to the flight device 100, the mail screen 45B is displayed on the touch screen display 2 when the flight device 100 is floating. The mail screen 45B displays the title, sender, and body of the mail in a size larger than the character size of the mail screen 45A.

In the example shown in FIG. 15, the case where the character size of the title, sender, and body of the mail screen 45B is made larger than that of the mail screen 45A will be described, but the present invention is not limited to this. For example, the mobile terminal 1 may make at least one character size of the title, sender, and text of the mail screen 45B larger than that of the mail screen 45A.

For example, when the flight device 100 is floating, the distance between the mobile terminal 1 connected to the flight device 100 and the user may be increased, or the flight device 100 may shake and become unstable. When the flight device 100 to which the mobile terminal 1 is connected is floating, the mobile terminal 1 increases the character size of the mail displayed on the touch screen display 2 to increase the visibility when the flight device 100 is floating. Can be suppressed.

FIG. 16 is a flowchart showing a processing procedure of an example of control related to the mail display function by the mobile terminal 1. The processing procedure shown in FIG. 16 is realized by the controller 10 executing the control program 9A. The processing procedure shown in FIG. 16 is repeatedly executed by the controller 10.

As shown in FIG. 16, the controller 10 of the mobile terminal 1 determines whether or not a mail display event has occurred (step S501). The mail display event occurs, for example, when a mail is selected in the mail reception list. When the controller 10 determines that the mail display event has not occurred (No in step S501), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that the mail display event has occurred (Yes in step S501), the controller 10 proceeds to the process in step S502.

The controller 10 determines whether or not the flight device 100 is floating (step S502). For example, when the controller 10 instructs the flight device 100 to float, the controller 10 determines that the flight device 100 is floating. For example, the controller 10 may determine whether or not the flight device 100 is floating based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like.

When the controller 10 determines that the flight device 100 is not floating (No in step S502), the controller 10 proceeds to the process in step S503. The controller 10 causes the designated mail to be displayed on the display 2A in a predetermined character size (step S503). When the controller 10 displays the mail, the controller 10 proceeds to the process in step S505.

When the controller 10 determines that the flight device 100 is floating (Yes in step S502), the controller 10 proceeds to step S504. The controller 10 enlarges the character size of the designated mail and displays it on the display 2A (step S504). For example, the controller 10 enlarges the character size of the mail to a character size larger than a predetermined size. When the controller 10 displays the mail, the controller 10 proceeds to the process in step S505.

The controller 10 determines whether or not to end the display event (step S505). When the controller 10 determines that the display event is not terminated (No in step S505), the controller 10 returns the process to step S505 already described. When the controller 10 determines that the display event is finished (Yes in step S505), the controller 10 proceeds to step S506. The controller 10 ends the display of the designated mail (step S506). When the controller 10 erases the mail displayed on the display 2A, the controller 10 ends the processing procedure shown in FIG.

An example of the map display function of the mobile terminal 1 will be described.
The navigation application 9G provides a navigation function for directions and the like. When connected to the flight device 100, the navigation application 9G can provide a function of changing the map display function depending on whether or not a predetermined condition is satisfied. The predetermined condition includes, for example, a condition that the flight device 100 is floating when connected to the flight device 100. The navigation application 9G can provide a function of navigating the flight device 100 in a state of being suspended in front of the user when connected to the flight device 100.

The setting data 9Z includes data indicating whether to enable or disable the function of changing the display of the map according to a predetermined condition when the mobile terminal 1 is attached to the flight device 100. The mobile terminal 1 can determine whether or not to change the map display function according to a predetermined condition by setting the setting data 9Z.

FIG. 17 is a diagram showing an example of switching the map display function by the mobile terminal 1. As shown in step S71 of FIG. 17, when the mobile terminal 1 is connected to the flight device 100, the map screen 47A is displayed on the touch screen display 2 when the flight device 100 is not floating. The map screen 47A is a screen for displaying a map image in a predetermined display size. As shown in step S72 of FIG. 17, when the mobile terminal 1 is connected to the flight device 100, the map screen 47B is displayed on the touch screen display 2 when the flight device 100 is floating. The map screen 47B is a screen for displaying an enlarged map of a map image on the same screen as the map screen 47A.

For example, when the flight device 100 is floating, the distance between the mobile terminal 1 connected to the flight device 100 and the user may be increased, or the flight device 100 may shake and become unstable. When the flight device 100 to which the mobile terminal 1 is connected is floating, the mobile terminal 1 enlarges the map displayed on the touch screen display 2 to reduce the visibility when the flight device 100 is floating. It can be suppressed. Since the mobile terminal 1 does not require the user to operate the switching of the map display function, it is possible to suppress a decrease in convenience even if the mobile terminal 1 floats on the flight device 100.

FIG. 18 is a flowchart showing a processing procedure of an example of control related to the map display function by the mobile terminal 1. The processing procedure shown in FIG. 18 is realized by the controller 10 executing the control program 9A. The processing procedure shown in FIG. 18 is repeatedly executed by the controller 10.

As shown in FIG. 18, the controller 10 of the mobile terminal 1 determines whether or not a map display event has occurred (step S601). The map display event occurs, for example, when the navigation screen is changed or switched. When the controller 10 determines that the map display event has not occurred (No in step S601), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that the map display event has occurred (Yes in step S601), the controller 10 proceeds to the process in step S602.

The controller 10 determines whether or not the flight device 100 is floating (step S602). For example, when the controller 10 instructs the flight device 100 to float, the controller 10 determines that the flight device 100 is floating. For example, the controller 10 may determine whether or not the flight device 100 is floating based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like.

When the controller 10 determines that the flight device 100 is not floating (No in step S602), the controller 10 proceeds to the process in step S603. The controller 10 causes the designated map to be displayed on the display 2A in a predetermined display size (step S603). When the controller 10 displays the map in a predetermined display size, the process proceeds to step S605.

When the controller 10 determines that the flight device 100 is floating (Yes in step S602), the controller 10 proceeds to the process in step S604. The controller 10 enlarges a map having a predetermined display size and displays it on the display 2A (step S604). When the controller 10 enlarges and displays the map beyond the predetermined display size, the process proceeds to step S605.

The controller 10 determines whether or not to end the display event (step S605). When the controller 10 determines that the display event is not terminated (No in step S605), the process returns to step S605 already described. When the controller 10 determines that the display event is finished (Yes in step S605), the controller 10 proceeds to the process in step S606. The controller 10 ends the display of the designated map (step S606). When the controller 10 erases the map displayed on the display 2A, the controller 10 ends the processing procedure shown in FIG.

FIG. 19 is a diagram showing another example of switching the map display function by the mobile terminal 1. As shown in FIG. 19, when the mobile terminal 1 is connected to the flight device 100, when the flight device 100 is floating, the mobile terminal 1 projects a map image onto the projection area E of the projector 23 of the mobile terminal 1. The projection area E is an area on which the projector 23 can project. The projection area E is an area of the ground near the user. The projection area E can be the area of the ground in the direction of travel of the user when the player is walking.

For example, when the flight device 100 is floating, the distance between the mobile terminal 1 connected to the flight device 100 and the user may be increased, or the flight device 100 may shake and become unstable. When the flight device 100 to which the mobile terminal 1 is connected is floating, the mobile terminal 1 displays a map image on the ground or the like by the projector 23, so that the visibility is reduced when the flight device 100 is floating. Can be suppressed. The mobile terminal 1 can improve the visibility during movement by the flight device 100 by causing the walking user to refer to the map projected on the ground.

FIG. 20 is a flowchart showing a processing procedure of another example of control regarding the map display function by the mobile terminal 1. The processing procedure shown in FIG. 20 is realized by the controller 10 executing the control program 9A. The processing procedure shown in FIG. 20 is repeatedly executed by the controller 10.

As shown in FIG. 20, the controller 10 of the mobile terminal 1 determines whether or not a map display event has occurred (step S601). When the controller 10 determines that the map display event has not occurred (No in step S601), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that the map display event has occurred (Yes in step S601), the controller 10 proceeds to the process in step S602.

The controller 10 determines whether or not the flight device 100 is floating (step S602). For example, when the controller 10 instructs the flight device 100 to float, the controller 10 determines that the flight device 100 is floating. For example, the controller 10 may determine whether or not the flight device 100 is floating based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like.

When the controller 10 determines that the flight device 100 is not floating (No in step S602), the controller 10 proceeds to the process in step S603. The controller 10 causes the designated map to be displayed on the display 2A in a predetermined display size (step S603). When the controller 10 displays the map in a predetermined display size, the process proceeds to step S614.

When the controller 10 determines that the flight device 100 is floating (Yes in step S602), the controller 10 proceeds to step S611. The controller 10 executes a specific process for identifying the position of the user based on the image taken by the camera (step S611). For example, the controller 10 that has executed the specific process takes a picture of the surroundings of the own aircraft or the flight apparatus 100 with the own aircraft or the camera of the flight apparatus 100 while rotating the flight apparatus 100 at a raised position. The controller 10 compares the captured image with the image of the user prepared in advance, and determines whether or not the user exists. When the user does not exist, the controller 10 stores in the storage 9 as a result that the position of the user could not be specified. When a user exists, the controller 10 calculates the position of the user based on information such as the current position of the own machine, the proximity sensor 5, the distance to the user measured by the distance sensor, and the like. When the specific process is completed, the controller 10 advances the process to step S612.

The controller 10 identifies the projection area E based on the identified user position (step S612). For example, the controller 10 specifies a region on the surface of an object near the user as a projection region E based on the position of the user, the position of the user, the position of the user, and the projection angle of the projector 23. The controller 10 projects a map having a predetermined display size onto the specified projection area E from the projector 23 (step S613). As a result, the mobile terminal 1 can project a map image from the projector 23 near the user.

The controller 10 determines whether or not to end the display event (step S614). When the controller 10 determines that the display event is not terminated (No in step S614), the controller 10 returns the process to step S614 already described. When the controller 10 determines that the display event is finished (Yes in step S614), the controller 10 proceeds to step S615. The controller 10 executes the termination process (step S615). The termination process includes a process of ending the projection by the projector 23. The termination process includes a process of erasing the map displayed on the display 2A. When the controller 10 finishes the end processing, the controller 10 ends the processing procedure shown in FIG.

An example of the navigation function of the mobile terminal 1 will be described.
The mobile terminal 1 can change the navigation function by executing the navigation application 9G according to a predetermined condition. For example, when the mobile terminal 1 detects a request for guidance by display, it provides a navigation function by displaying navigation information. For example, when the mobile terminal 1 is connected to the flight device 100 and detects a request for navigation due to the floating device 100, the function is changed to the navigation function due to the floating device 100.

FIG. 21 is a flowchart showing a processing procedure of an example of control related to the navigation function by the mobile terminal 1. The processing procedure shown in FIG. 21 is realized by the controller 10 executing the control program 9A, the flight object cooperation program 9B, and the navigation application 9G. In the processing procedure shown in FIG. 21, control related to collision avoidance with people and obstacles around the flight device 100 is omitted.

As shown in FIG. 21, the controller 10 of the mobile terminal 1 identifies a route to the set destination (step S701). For example, the mobile terminal 1 identifies a route based on the current position of the own device, the position of the destination, and map information. The controller 10 determines whether or not the flight is guided by the flight device 100 (step S702). For example, when the user requests guidance by the flight device 100, the controller 10 determines that the guidance is by the flight device 100.

When the controller 10 determines that the guidance is not by the flight device 100 (No in step S702), the controller 10 determines that the guidance is by the display of the mobile terminal 1, and proceeds to the process in step S703. The controller 10 executes the guidance process by displaying the guidance screen without flying the flight device 100 (step S703). The guidance process includes a process of displaying the guidance information on the display 2A until the destination is reached. The guidance process includes a process of notifying information for guiding the user when the current position of the own machine satisfies the guidance condition. Guidance conditions include, for example, conditions such as detecting that the vehicle is approaching a left or right turn, detecting that the vehicle is approaching a landmark, and detecting that the vehicle is approaching a destination. The guidance process ends when the destination is reached. When the guidance process is completed, the controller 10 ends the process procedure shown in FIG.

If the controller 10 determines that the guidance is due to the flight device 100 (Yes in step S702), the controller 10 proceeds to step S704. The controller 10 transmits control information instructing the flight device 100 to ascend (step S704). For example, the controller 10 transmits control information including the ascending height and the like to the flight device 100 via the communication unit 6. As a result, the flight device 100 floats to a predetermined height with the mobile terminal 1 attached. After that, the controller 10 advances the process to step S705.

The controller 10 determines the flight speed, flight direction, and flight distance of the flight device 100 (step S705). For example, the controller 10 keeps the distance between the flight device 100 and the user at a constant distance based on the route to the destination, the current position of the aircraft, the moving speed of the user, and the distance to the user. In addition, the flight speed, flight direction, flight distance, and the like of the flight device 100 are determined. For example, when the distance from the flight device 100 to the user is large, the controller 10 determines the flight speed, flight direction, flight distance, and the like for stopping flight without moving at that location. After that, the controller 10 advances the process to step S706.

The controller 10 transmits control information instructing the flight speed, flight direction, and flight distance (step S706). For example, the controller 10 transmits control information including flight speed, flight direction, and flight distance to the flight device 100 via the communication unit 6. As a result, the flight device 100 flies to the designated position with the mobile terminal 1 attached or hoveres on the spot based on the control information. After that, the controller 10 advances the process to step S707.

The controller 10 determines whether or not the flight device 100 has arrived at the destination based on the current position of the own aircraft (step S707). When the controller 10 determines that the flight device 100 has not arrived at the destination (No in step S707), the controller 10 returns the process to step S705 already described. When the controller 10 determines that the flight device 100 has arrived at the destination (Yes in step S707), the controller 10 proceeds to the process in step S708.

The controller 10 transmits control information instructing the flight device 100 to end the flight (step S708). For example, the controller 10 transmits control information instructing landing to the flight device 100 via the communication unit 6. As a result, the flight device 100 lands with the mobile terminal 1 attached. After that, the controller 10 ends the processing procedure shown in FIG.

By guiding the user to the destination by the flight device 100 that has been flown, the mobile terminal 1 eliminates the need for walking while looking at the display 2A, and can improve the safety of the user to be guided. For example, when the mobile terminal 1 guides the user to the destination by the flight device 100, the mobile terminal 1 projects a map image, guidance information, and the like from the projector 23 onto the projection area E in front of the user. Convenience can be improved.

An example of the health function of the mobile terminal 1 will be described.
The control program 9A provides a function for determining whether or not the user who owns the own machine is moving by a predetermined movement method based on the value of the acceleration detected by the acceleration sensor 15.

The calculation application 9H is used, for example, to calculate the energy consumption of the user. The calculation application 9H provides, for example, a function for calculating the user's consumption by applying the user's movement speed to the relational expression between the user's movement speed (number of steps) and the energy consumption. The energy consumption of the user may be calculated in consideration of the movement method of the user. For example, when the user walks the same number of steps, the calculation application 9H may change the relational expression of the energy consumption depending on whether the user walks or runs. The energy consumption of the user may be calculated in consideration of the mobile environment of the user. For example, when the user walks the same number of steps, the calculation application 9H changes the relational expression of the energy consumption depending on whether the road the user is walking is flat or uphill. May be good. As this energy consumption, for example, thermodynamic calories (cal) are adopted as "measurement of the amount of heat consumed by humans or animals or the amount of heat consumed by metabolism by humans or animals" based on the Measurement Law of Japan. To. The measurement of energy consumption is not limited to this, and joules (J) may be adopted based on CGPM (Conference General des Poids et Measures).

The target calculated by the calculation application 9H is not limited to the energy consumption of the user, and may calculate the user's exercise, the number of steps when moving by walking, and the like. "Exercise" is a unit that expresses the amount of physical activity. Exercise is the amount of exercise calculated by multiplying Mets, which will be described later, by the duration of physical activity. This Mets is a unit that expresses the strength of physical activity. The intensity of this physical activity varies depending on the type of physical activity. The METs are set for each method of moving the user, for example. Mets may be used in the calculation of energy consumption, which is an activity factor indicating the amount of activity. In Mets, it is expressed as a ratio to the intensity of physical activity at rest. For example, it is said that sitting and resting corresponds to 1 MET, and normal walking corresponds to 3 METs. In other words, the intensity of physical activity in normal walking is three times the intensity of physical activity at rest.

The mobile terminal 1 can provide a health function for calculating the number of steps, consumption, etc. of a user by the calculation application 9H when the moving state of the own device is a walking state or a running state. When the own aircraft connected to the flight device 100 satisfies a predetermined condition, the mobile terminal 1 changes the health function of the own aircraft.

For example, when the mobile terminal 1 is attached to the flight device 100, if the movement speed is detected based on the vibration caused by the movement of the user, the vibration cannot be detected. Therefore, the mobile terminal 1 changes the health function when the user carries his / her own aircraft when the predetermined conditions are attached to the flight device 100 and the flight device 100 is floating. The mobile terminal 1 can estimate the movement speed (number of steps), consumption amount, etc. of the user based on the movement amount of the flight device 100. Since the mobile terminal 1 can estimate the movement speed (number of steps), consumption amount, etc. of the user even when moving by flight while being connected to the flight device 100, it is possible to suppress a decrease in convenience. .. As a result, the mobile terminal 1 can leave an activity log such as the number of steps and consumption even if the user does not wear his / her own device. The mobile terminal 1 can increase the opportunities of users who connect their own aircraft to the flight device 100 and use them, and can provide a novel service.

FIG. 22 is a flowchart showing a processing procedure of an example of control of the health function by the mobile terminal 1 in flight. The processing procedure shown in FIG. 22 is realized by the controller 10 executing the control program 9A and the calculation application 9H. The processing procedure shown in FIG. 22 is repeatedly executed by the controller 10 when the mobile terminal 1 is attached to the flight device 100.

As shown in FIG. 22, the controller 10 of the mobile terminal 1 determines whether or not the flight device 100 has started floating (step S801). The floating of the flight device 100 includes, for example, the ascent and flight of the flight device 100. For example, when the controller 10 instructs the flight device 100 to start floating, the controller 10 determines that the flight device 100 has started floating. For example, the controller 10 may determine the start of floating of the flight device 100 based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like. When the controller 10 determines that the flight device 100 has not started floating (No in step S801), the controller 10 ends the processing procedure shown in FIG. When the controller 10 determines that the flight device 100 has started floating (Yes in step S801), the controller 10 proceeds to step S802.

The controller 10 acquires the position information of its own aircraft mounted on the floating flying object (step S802). For example, the controller 10 acquires the position information received by the GPS receiver 20 when the distance between the flight device 100 and the user is closer than a predetermined distance based on the detection result of the proximity sensor 5 or the like. As a result, the controller 10 can acquire the current position close to the user's position. The controller 10 stores the acquired position information in the storage 9 in chronological order (step S803). The controller 10 calculates the amount of movement when the flight device 100 is floating based on the position information of the storage 9 (step S804). For example, the controller 10 calculates the movement amount based on the position information of the storage 9 from the start of the movement of the flight device 100 to the present. The controller 10 updates the movement amount stored in the storage 9 based on the calculated movement amount (step S805). After that, the controller 10 advances the process to step S806.

The controller 10 determines whether or not the flight device 100 has finished floating (step S806). For example, when the controller 10 instructs the flight device 100 to end floating, the controller 10 determines that the flight device 100 has finished floating. For example, the controller 10 may determine the end of floating of the flight device 100 based on the detection results of the barometric pressure sensor 18, the acceleration sensor 15, and the like. When the controller 10 determines that the flight device 100 has not finished floating (No in step S806), since the flight device 100 continues to float, the process returns to step S802 already described. When the controller 10 determines that the flight device 100 has finished floating (Yes in step S806), the controller 10 proceeds to the process in step S807.

The controller 10 estimates the number of steps of the user based on the amount of movement of the flight device 100 from the start of floating to the end of floating and the stride of the user (step S807). For example, the controller 10 may calculate the consumption amount of the user by applying the number of steps of the user to the estimated relational expression between the number of steps of the user and the energy consumption amount. The controller 10 stores the estimated number of steps of the user in the storage 9 (step S808). For example, the controller 10 may store the estimated number of steps in the storage 9 as the number of steps of the user when the flight device 100 is floating. For example, the controller 10 may add the estimated number of steps to the number of steps when the user carries his / her own machine. After that, the controller 10 ends the processing procedure shown in FIG.

In the above embodiment, the mobile terminal 1 has been described as an example of the mobile electronic device, but the mobile electronic device according to the attached claim is not limited to the mobile terminal 1. The portable electronic device according to the attached claim may be a portable electronic device other than the mobile terminal 1. Portable electronic devices include, but are not limited to, for example, mobile phones, smartphones, smart watches, portable personal computers, head-mounted displays, digital cameras, media players, e-readers, navigators, and game consoles.

In order to fully and clearly disclose the technology according to the attached claims, characteristic embodiments have been described. However, the accompanying claims should not be limited to the above embodiments, and all modifications and alternatives that can be created by those skilled in the art within the scope of the basic matters set forth herein. It should be configured to embody a unique configuration.

1 Mobile terminal 2A Display 2B Touch screen 2 Touch screen display 3 Button 4 Illumination sensor 5 Proximity sensor 6 Communication unit 7 Receiver 8 Microphone 9 Storage 9A Control program 9B Aircraft cooperation program 9C User search data 9D Image authentication data 9E Voice authentication Data for 9F Mail application 9G Navigation application 9H Calculation application 9Z Setting data 10 Controller 11 Speaker 12 Camera 13 Camera 14 Connector 15 Acceleration sensor 16 Orientation sensor 17 Angle speed sensor 18 Pressure sensor 19 Pressure sensor 20 GPS receiver 21 Battery 22 Charging unit 23 Projector 100 Flying device 100F Air vehicle

Claims (15)

An air vehicle including a portable electronic device and a flight device to which the portable electronic device is detachably connected.
An air vehicle that changes a predetermined function of the portable electronic device when the portable electronic device detects that the flight device is flying. The flying object according to claim 1 , wherein the flight power of the flight device is controlled according to the function of the portable electronic device after the change. The flying object according to claim 1 or 2 , wherein the predetermined function is a display function. The flying object according to claim 1 or 2 , wherein the predetermined function is a charging function. The flying object according to claim 1 or 2 , wherein the predetermined function is a notification function. The flying object according to claim 1 or 2 , wherein the predetermined function is an input function. The flying object according to claim 1 or 2 , wherein the predetermined function is a health function. The flying object according to claim 1 or 2 , wherein the predetermined function is a navigation function. A portable electronic device connected to a flight device
A communication unit that communicates with the flight device,
With a controller that performs a given function,
The controller
A portable electronic device that changes its predetermined function when it detects that it is flying by the flight device. The portable electronic device according to claim 9 , wherein the predetermined function is a display function. The portable electronic device according to claim 9 , wherein the predetermined function is a charging function. The portable electronic device according to claim 9 , wherein the predetermined function is a notification function. The portable electronic device according to claim 9 , wherein the predetermined function is an input function. The portable electronic device according to claim 9 , wherein the predetermined function is a health function. The portable electronic device according to claim 9 , wherein the predetermined function is a navigation function.

Images