JP6603438B2 - Information processing apparatus, program, and method - Google Patents

Description

  The technology disclosed in the present application relates to an information processing apparatus that executes information processing related to an unmanned aerial vehicle.

In recent years, the use of unmanned aerial vehicles has expanded, and the number of users has increased. As an application of unmanned aerial vehicles, in addition to those using the carrying capacity, unmanned aircraft are equipped with a camera and photographed. For example, Japanese Patent Laying-Open No. 2006-225874 (Patent Document 1) proposes a technique for zooming an image of a camera provided in an unmanned aerial vehicle (mobile device).
However, since an unmanned aerial vehicle flies, it is not easy to operate and requires a certain amount of training in order to control it safely and freely. For example, Japanese Patent Laying-Open No. 2006-225872 (Patent Document 2) proposes a new technique for improving remote operation because it is difficult to move an unmanned aircraft.

JP, 2006-225874, A JP 2006-225872 A

However, even with the conventional techniques described in the above technical documents, a technique for measuring the flight state of an unmanned aerial vehicle as desired by a user has not been realized.
Therefore, according to various embodiments of the present invention, an information processing apparatus, program, or method having a function of allowing a user to measure the flight state of an unmanned aerial vehicle is provided.

  An embodiment of the present invention is directed to an imaging unit, an image processing unit that performs image processing on part or all of an image based on the imaging unit, and the image based on part or all of information obtained from the image processing unit. It is an information processing apparatus provided with the position comparison determination part which compares and determines the information which concerns on the position of the unmanned aircraft in the inside, and a predetermined value or range.

  In an embodiment of the present invention, the information processing apparatus may further include an evaluation unit that evaluates a flight state of the unmanned aircraft based on part or all of the information obtained by the position comparison determination unit. Good.

According to an embodiment of the present invention, the flight state of an unmanned aerial vehicle can be measured.

FIG. 1 is a schematic diagram showing the configuration of an embodiment according to the present invention. FIG. 2 is a schematic diagram of an unmanned aerial vehicle showing a part of an embodiment according to the present invention. FIG. 3 is a schematic view of an unmanned aerial vehicle showing a part of an embodiment according to the present invention. FIG. 4 is a diagram related to the outline of a transmitter showing a part of an embodiment according to the present invention. FIG. 5 is a schematic diagram of an information processing apparatus related to a part of one embodiment according to the present invention. FIG. 6 is a schematic diagram of an information processing apparatus related to a part of one embodiment according to the present invention. FIG. 7 is a schematic functional diagram of an information processing apparatus related to a part of one embodiment according to the present invention. FIG. 8 is a schematic diagram related to a part of one embodiment according to the present invention. FIG. 9 is a schematic diagram related to a part of one embodiment according to the present invention. FIG. 10 is an example in which information relating to color, which is a part of one embodiment according to the present invention, is recorded. FIG. 11 is a schematic diagram of a fuselage aspect of an unmanned aerial vehicle related to a part of one embodiment according to the present invention. FIG. 12 is a schematic diagram of a fuselage aspect of an unmanned aerial vehicle related to a part of one embodiment according to the present invention. FIG. 13 is a schematic view of an airframe aspect of an unmanned aerial vehicle related to a part of one embodiment according to the present invention. FIG. 14 is a schematic diagram of a fuselage aspect of an unmanned aerial vehicle related to a part of one embodiment according to the present invention. FIG. 15 is a schematic diagram of a fuselage aspect of an unmanned aerial vehicle related to a part of one embodiment according to the present invention. FIG. 16 is a schematic diagram of a fuselage aspect of an unmanned aerial vehicle related to a part of one embodiment according to the present invention. FIG. 17 is an example of a storage table related to a part of one embodiment according to the present invention. FIG. 18 is an example of a storage table related to a part of one embodiment according to the present invention. FIG. 19 is an example of a storage table related to a part of one embodiment according to the present invention. FIG. 20 is an example of a displayed image related to a part of one embodiment according to the present invention. FIG. 21 is an example of a storage table related to a part of one embodiment according to the present invention. FIG. 22 is a flowchart relating to a part of one embodiment according to the present invention. FIG. 23 is a flowchart relating to a part of one embodiment according to the present invention. FIG. 24 is a flowchart relating to a part of one embodiment according to the present invention. FIG. 25 is a flowchart relating to a part of one embodiment according to the present invention. FIG. 26 is a flowchart relating to a part of one embodiment according to the present invention. FIG. 27 is an example of a storage table related to a part of one embodiment according to the present invention. FIG. 28 is a flowchart relating to a part of one embodiment according to the present invention. FIG. 29 is an example of a displayed image related to a part of one embodiment according to the present invention. FIG. 30 is an example of a displayed image related to a part of one embodiment according to the present invention. FIG. 31 is an example of a displayed image related to a part of one embodiment according to the present invention.

  Hereinafter, various embodiments according to the present invention will be described with reference to the accompanying drawings.

1. Overall System FIG. 1 shows a state in which an unmanned aerial vehicle is imaged as an embodiment. Reference numeral 101 denotes an unmanned aircraft that is flying. Reference numeral 102 denotes an information processing apparatus that images the unmanned aircraft 101. Reference numeral 103 denotes a transmitter that controls the flight of the unmanned aircraft 101. The information processing apparatus 102 may be detachably fixed to the transmitter 103. The transmitter 103 may include an adjustment mechanism that can adjust the position of the information processing device 102 so that the information processing device 102 can adjust the angle at which the unmanned aircraft 101 can be imaged when the unmanned aircraft 101 flies. In FIG. 1, the target paper 104 is used to make the position of the unmanned aircraft more easily understood.

  The size and capability of the unmanned aerial vehicle 101 may be anything. For example, the unmanned aerial vehicle 101 may be several centimeters or a large one having a size of 1 meter or more.

  Further, the imaging function of the information processing apparatus 102 may be any type. For example, when the information processing apparatus 102 is a smartphone and the size of the unmanned aircraft 101 is about 2 centimeters to 6 centimeters, the information processing apparatus 102 may change from the information processing apparatus 102 to the unmanned aircraft. The distance 101 is preferably about 1 meter to about 3 meters. In the case of unmanned aerial vehicles, in relation to educational purposes, large aircraft (20 cm or more) can be stabilized by polishing flight technology with small unmanned aerial vehicles (for example, about 2 to 6 centimeters). There is an advantage to be able to fly.

2. Unmanned Aircraft and Transmitter FIG. 2 shows an example of an unmanned aerial vehicle used in the present embodiment, and is a drawing of the unmanned aircraft in flight as viewed from above. The unmanned aerial vehicle includes four rotary blades 203, and the upper surface of the unmanned aerial vehicle body is painted in two colors. Specifically, when FIG. 2 is viewed from the paper surface, the horizontal line on the upper side 201 indicates substantially blue. When viewing FIG. 2 from the plane of the drawing, the vertical line on the lower side 202 shows a substantially orange color.

  FIG. 3 is a side view of the unmanned aerial vehicle shown in FIG. The top of the unmanned aircraft is painted in two colors. The horizontal line on the upper left 301 indicates a substantially blue color. The vertical line 302 on the right side shows a substantially orange color. The lower part of the unmanned aerial vehicle is white.

  FIG. 4 shows an example of a transmitter for an unmanned aerial vehicle used in the present embodiment. The transmitter 401 is configured to be detachably equipped with an information processing device 402 at the top. In FIG. 4, the transmitter includes a smartphone as an information processing device so that it can be attached and detached.

  Although not shown in the figure, a mode in which the user operates the unmanned aerial vehicle while wearing a head-mounted display and watching an image on the head-mounted display may be employed.

3. Each configuration of the information processing apparatus In the present embodiment, in the case of a system that does not include a network, the information processing apparatus 10 includes a bus 11, an imaging unit 12, a calculation unit 13, a storage unit 14, an input unit 15, as shown in FIG. A display unit 16 and a communication IF 17 can be provided.

  The bus 11 has a function of transmitting information among the imaging unit 12, the calculation unit 13, the storage unit 14, the input unit 15, the display unit 16, and the communication IF 17.

The imaging unit 12 has a function of imaging a subject. For example, a video camera or a camera is mentioned. The captured data may be a moving image or a still image.
The calculation unit 13 has a function of calculating. For example, a CPU and MPU can be mentioned. In addition, the calculation unit 13 may include a graphics processing unit, a digital signal processor, and the like. The calculation unit 13 may acquire the program from the recording unit 14 and execute the program, or may execute the calculation using a dedicated circuit.

  The storage unit 14 has a function of recording information.

  The input unit 15 has a function of inputting information. Examples include input devices such as touch panels and keys. An input device such as a keyboard or a mouse may be used.

  The display unit 16 has a function of displaying information. A liquid crystal display, a plasma display, an organic EL display, etc. may be used. Moreover, you may provide the input part 15 in part or all like a touch screen panel.

  The communication IF 17 has a function of transmitting information directly or indirectly with an external information processing apparatus. You may have a calculation function or a memory | storage function only for the function to communicate. Further, it may be wired or wireless. Since it is only necessary to communicate indirectly with the outside, an interface that can be connected to an external storage may be used. The interface may be a serial type or a parallel type. For example, the format may be USB, IEEE 1394, Ethernet (registered trademark), SCSI, PCI, or the like. The wireless communication function may be a wireless LAN, WLAN, Wi-Fi, Bluetooth (registered trademark), or the like.

  The information processing apparatus 10 may be a general-purpose computer or a dedicated computer. The information processing apparatus 10 may be a mobile terminal device such as a mobile phone, a smartphone, a PDA, a PHS, a mobile information terminal, a mobile device, a tablet PC, a handheld PC, a smart book, or a head-mounted display.

  In addition, some of the functions provided in the information processing apparatus 10 may be provided in another device. For example, a part of the imaging unit 12, the calculation unit 13, the storage unit 14, the input unit 15, and the display unit 16 may be provided by another device. When another device has the partial function, the information processing apparatus 10 may transmit information related to the other device, or may transmit information related to the other device.

  FIG. 6 shows an information processing apparatus 20 that can exchange information with the information processing apparatus 10. The information processing apparatus 20 can include a bus 21, a calculation unit 22, a storage unit 23, an input unit 24, an output unit 25, and a communication IF 27. The function of each part of the information processing apparatus 20 can be considered in the same way as the function of each part of the information processing apparatus 10. That is, the function of the bus 21 corresponds to the function of the bus 11, the function of the calculation unit 22 corresponds to the function of the calculation unit 13, the function of the storage unit 23 corresponds to the function of the storage unit 14, and the function of the input unit 24. Corresponds to the function of the input unit 15, the function of the output unit 25 includes the function of the display unit 16, the function of the communication IF 17 corresponds to the function of the communication IF 27, and is configured by hardware capable of realizing each function. However, the output unit 25 is not limited to the display function, and may have a normal output function.

  Although the information processing device 20 is assumed to be connected to the information processing device 10 via the Internet 28, the information processing device 20 may be configured to be able to transmit information by a communication medium corresponding to the communication IF 17 described above.

  The information processing apparatus 20 may be a server. When the information processing apparatus 20 is a server, the calculation unit 22 may have a high-performance calculation function, and the storage unit 23 may have a high-function storage function.

  The relationship between the information processing apparatus 10 and the information processing apparatus 20 may be a P2P system, a grid system, a cloud system, or a mixture thereof, in addition to the client server system.

  As long as one embodiment according to the present invention is a hardware system capable of realizing any of the software functions described below, one of the various hardware systems described above or hardware other than the various hardware systems described above It can also be applied in the system.

4). Each unit in the information processing apparatus and its function Using the hardware of the above-described information processing apparatus, an embodiment of the information processing apparatus includes a storage unit 700, an image acquisition unit 701, an image processing unit 702, as shown in FIG. A position comparison / determination unit 703, an evaluation unit 704, and a display control unit 705 are included. The information processing apparatus 10 may include each unit, but the information processing apparatus 20 may include a part thereof.

4-0. Storage unit 700
The storage unit 700 has a function of storing information. The storage unit 700 stores various types of information accompanying the functions of the units described below. The storage unit 700 may store information related to the embodiment according to the present application and may store information for other purposes. Information stored in the storage unit 700 may be updated as appropriate. Depending on the amount or timing of information, the information may be directly transmitted from each unit to each other unit without using the storage unit 700.

4-1. Image acquisition unit 701
The image acquisition unit 701 has a function of acquiring an image. An image may be acquired from the image capturing unit, or an image may be acquired via a communication IF or the like. The image may be a moving image or a still image.

  The format of the image is not particularly limited. Examples include JPEG, TIFF, BITMAP, PNG, PDF, GIF, EPS, and the like.

4-2. Image processing unit 702
The image processing unit 702 processes an image as preprocessing in order to acquire information related to the position of the unmanned aircraft.
In the present embodiment, when the image is an RGB color system, it is converted into an HSV color space (HSV model). The HSV model is composed of three components: hue, saturation, and brightness. In this conversion, for example, H is the following expression, and conversion is performed by S = Max−Min and B = Max.

Here, among the three values R, G, and B, the maximum value is Max and the minimum value is Min. HSV also has the advantage of being able to be identified stably because the hue does not change, although the brightness and saturation may change even when the weather is different outside.

Note that if the image needs to be preprocessed when the image acquisition unit 701 acquires information, the image processing unit 702 may not perform the process.
The image processing unit 702 may perform noise processing. A known technique may be used for the noise processing itself.

4-3. Position comparison / determination unit 703
The position comparison / determination unit 703 has a function of comparing and determining information related to the position of the unmanned aircraft and a predetermined value or a predetermined range.

  Information such as information on the position of the unmanned aircraft, a predetermined value, or a predetermined range may be stored in the storage unit 700.

  The information related to the position of the unmanned aircraft may be information that allows the position of the unmanned aircraft to be acquired directly or indirectly. For example, the information related to the position of the unmanned aircraft may be composed of a plurality of numerical values related to the position of the unmanned aircraft, or may be a specific value related to the position of the unmanned aircraft. The former may be, for example, XY coordinates indicating the position of the unmanned aircraft when an unmanned aircraft exists in the XY coordinates. The coordinates may be the XY coordinate value of the center of the unmanned aircraft or the XY coordinate value of the center of gravity. Moreover, the set of the value of the XY coordinate which shows the rectangle containing the unmanned aircraft in an image may be sufficient. For example, it may be indicated by four XY coordinates of the smallest rectangle surrounding the unmanned aircraft. Specifically, (minimum value X, minimum value Y), (maximum value X, minimum value Y), (minimum value X, maximum value Y), (maximum value X, maximum value Y), etc. are mentioned.

Each coordinate may be indicated by a value having a certain width. For example, X = (minimum value X, maximum value X), Y = (minimum value Y, maximum value Y), and the like.
Moreover, the XY coordinates of a specific part of the unmanned aerial vehicle may be used. For example, the position of a specific mark in the unmanned aircraft may be specified by XY coordinate values.

  Furthermore, a method may be used in which the image is divided into several parts, and a number is assigned to one of the parts, so that the position of the unmanned aircraft is specified by one of the numbers. For example, as shown in FIG. 8, the image may be divided into positions 1 to 16 and information relating to the position of the unmanned aircraft may be acquired based on the number related to the section overlapping with the unmanned aircraft. In this case, when an unmanned aerial vehicle overlaps with a plurality of portions, the position may be specified by a number relating to a wide area section that most overlaps an image of the unmanned aerial vehicle.

  In short, it is only necessary to acquire information related to the position of the unmanned aircraft in the image.

  The predetermined value or the predetermined range may be any value as long as at least a part of the predetermined value or the predetermined range can be compared with at least a part of the information related to the position of the unmanned aircraft. Good. For the purpose of comparison with information relating to the position of the unmanned aircraft, the corresponding value may be a single value or a plurality of values. For example, the predetermined value may be a single value such as Y = 3 in the XY coordinates. Further, when the predetermined value is a plurality of values, it may be a plurality of values such as X = 4 and X = 8 in the XY coordinates.

  The predetermined range may be specified by an arbitrary method. For example, in the XY coordinates, a rectangle composed of four points (X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2) may be used. In the XY coordinate system, a form such as a radius of 2 at the center (3, 4) of the circle may be used. Furthermore, a designation method such as X <= 5 may be used.

  Moreover, when it is determined by a predetermined value or a predetermined range, it is expressed based on various coordinate systems such as an oblique coordinate system, a polar coordinate system, and a generalized coordinate system in addition to the XY coordinate system (orthogonal coordinate system). Alternatively, the value used may be expressed by an arbitrary value that can be handled by a computer such as a natural number, an integer, a decimal number, a fraction, a rational number, or the like.

  The predetermined value or the predetermined range may be a value that can be compared, and may be determined in advance or may be a value determined at the time of comparison. The predetermined value or the predetermined range may be a value calculated by a predetermined function. The predetermined function may be a function having a value related to execution of the application as an argument. For example, it may be a value calculated by a predetermined function having arguments such as the total flight time of the user, the flight time of the current stage, the score acquired by the user, the difficulty level, etc., as will be described later.

  Information relating to the predetermined value or the predetermined range may be displayed on the display unit. For example, when the predetermined value is a predetermined range and is a rectangle composed of four points (X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2), A rectangle may be displayed on the screen.

  Here, as an example, for the sake of simplicity, the position of the unmanned aircraft is a predetermined range (30 <) in an image that can be measured with XY coordinates (0 <= X <= 150, 0 <= Y <= 100). = X <= 70, 20 <= Y <= 130) An example will be described in which determination is made by comparison.

  FIG. 9 shows a predetermined range 902 in the image 901. Unmanned aerial vehicle 903 indicates a case within a predetermined range, and unmanned aircraft 904 indicates a case outside a predetermined range.

  The unmanned aerial vehicle of this example shown in FIGS. 2 and 3 was coated with approximately orange and approximately blue. In the HSV color space, the substantially orange color has a value of 30 for H, 100 of S, and 90 for V, and substantially blue has a value of 215 for H, 71 for S, and 36 for V. Therefore, the position comparison / determination unit 703 compares information of each pixel within a predetermined range (30 <= X <= 70, 20 <= Y <= 130) with the above color, and the color value of each pixel is determined. It is determined whether it is the value.

  In the case where the color of the unmanned aerial vehicle is as described above, the position of the unmanned aircraft can be more easily specified when the background is white. That is, since the contrast between white and the above-mentioned color is conspicuous, there is an advantage that erroneous determination is reduced when information of each pixel is compared with the above-described color. For the white background, it is conceivable that, for example, white paper, cloth, a cover, or the like is installed on the information processing apparatus provided in the transmitter and the unmanned aircraft.

  When the value of each pixel is the color, since a part of the unmanned aircraft is found within the searched predetermined range, the position comparison determination unit 703 determines that the position of the unmanned aircraft is within the predetermined range. To do. That is, in the example of the present embodiment, in determining that the unmanned aircraft is within the predetermined range, it does not require that the entire unmanned aircraft is within the predetermined range. If it is within the predetermined range, it is determined that the unmanned aircraft is within the predetermined range.

  On the other hand, if the position comparison / determination unit 703 does not determine the same color as the color corresponding to the unmanned aerial vehicle even though the position comparison / determination unit 703 has searched within the predetermined range, the position comparison / determination unit 703 It is determined that it is not within the range.

  In the above example, the substantially orange and substantially blue values are uniquely specified, and when the values of the pixels searched by the position comparison determination unit 703 are the same, it is assumed that they are part of an unmanned aircraft. However, the determination may be made with a certain width between the values of substantially orange and blue. For example, a case determined as substantially orange may be calculated as H being 0 to 24, S being 144 to 255, and V being 102 to 255, or substantially blue being H being 0 to 100 and S being 35. Or 255, and V may be calculated as 30 to 255. In this case, the width of each color may be changed so that the color to be determined can be distinguished from other colors.

  The position comparison determination unit 703 may determine the color based on information about the color stored in the storage unit 700. For example, as shown in FIG. 10, information about colors may be held. FIG. 10 shows the stored information corresponding to the above-described approximately orange color and approximately blue color. The lower limit value and the upper limit value of acceptable color information of each color are recorded, and these information and the corresponding color information of each pixel are compared.

Further, the position comparison / determination unit 703 replaces the search within the predetermined range and out of the predetermined range (0 <= X <30, 70 <X <= 100, 0 <Y <20, 130 <Y <150). The pixel information may be compared with the above color to determine whether the color value of each pixel is the value.
In other words, in this case, if a part of the unmanned aircraft is outside the predetermined range, it is determined that the unmanned aircraft is outside the predetermined range even if a part of the unmanned aircraft is within the predetermined range. Yes.

As information related to unmanned aerial vehicles described here, a method of searching for a part of an unmanned aerial vehicle in an image, obtaining color information, and comparing (before obtaining a color of a part not searched) is The area within or outside the predetermined range at the maximum, that is, the search for only the pixels within the predetermined range or the search for the number of pixels only outside the predetermined range may be used. Compared to all searches, there is an advantage that the number of search pixels (area) is small.
In particular, when the position acquisition determination unit 703 needs to frequently acquire information related to the position of the unmanned aircraft, there is an advantage that processing can be performed even in a portable terminal device with low processing capability without increasing the processing load.

  In the above example, the search is terminated when it is determined that a part of the color of the unmanned aircraft exists within the predetermined range. A part or all of the other parts within the range may be further searched. In this case, for example, when the entire unmanned aircraft exists within a predetermined range, there is an advantage that information on the entire unmanned aircraft can be acquired. More specifically, for example, when two colors are used as the color of the unmanned aircraft, the direction of the unmanned aircraft described later may be determined.

  Similarly, in the example where the search is terminated when it is determined that a part of the color of the unmanned aircraft exists outside the predetermined range, a part of the other part outside the predetermined range or All may be further searched. In this case, for example, when the entire unmanned aircraft exists outside a predetermined range, there is an advantage that information on the entire unmanned aircraft can be acquired.

  However, each of the above methods only searches for some or all of the predetermined range or outside the predetermined range. In some cases, the entire information of the aircraft is not acquired.

  A method for searching both at least a part of the predetermined range and at least a part outside the predetermined range in the image will be described later.

  Further, the position comparison / determination unit 703 may compare the areas of the search portions within the predetermined range and the predetermined range before the above-described search, and search for the one with the smaller area of the search portion.

  In addition, although the example which compares each pixel independently was used above, it is not restricted to this. You may compare with the information of each color based on several pixels. For example, an average value of a plurality of pixels may be compared with information on each color, or may be compared with information on each color with a weighted numerical value of the plurality of pixels.

  Note that noise processing may be performed before searching for each color. For example, if approximately orange or approximately blue pixels are present alone in the image, there may be noise. This is because unmanned aerial planes are applied in a substantially orange or substantially blue area, so that a single pixel does not have that color. Therefore, when a substantially orange or substantially blue pixel exists alone as described above, the pixel may be changed to, for example, white other than substantially orange or substantially blue as noise processing.

  In the above example, the case where the pixel exists alone has been described. However, even when there are two or more pixels, the noise processing may be changed to a color other than substantially orange or substantially blue. The number or position of two or more pixels can be determined by the number of pixels or position that can be determined not to be part of an unmanned aerial vehicle. The above is only an example of noise processing, and other known noise processing techniques may be applied.

  Further, the position comparison / determination unit 703 does not need to search for all the pixels of the image as in the above-described embodiment, and based on the improved search strategy, does not search for a part of the comparison, The positional relationship may be determined.

  Although the above-described position comparison determination unit 703 performs the acquisition of the position of the unmanned aircraft and the comparison with the predetermined range, it may be performed as another step. That is, the position comparison determination unit 703 may include an acquisition unit 7031, a comparison unit 7032, and a determination unit 7033. Specifically, the acquisition unit 7031 mainly acquires information related to the position of the unmanned aircraft, the comparison unit 7032 mainly compares the information with a predetermined value, and the determination unit 7033 compares the information. It may be determined. Hereinafter, an embodiment in which the position comparison determination unit 703 includes an acquisition unit 7031, a comparison unit 7032, and a determination unit 7033 will be described.

4-3-1. Acquisition unit 7031: The example acquisition unit 7031 based on color information acquires information related to the position of the unmanned aircraft. The information related to the position of the unmanned aircraft may be acquired by any means. Here, an example in which information is acquired based on the color information will be described first. Moreover, the example which specifies the unmanned aircraft in an image is demonstrated as an example which acquires the information which concerns on the position of an unmanned aircraft.

  If the example of substantially orange and substantially blue is used as the color to be used, the acquisition unit 7031 searches each pixel of the image and determines whether each pixel has the above color. More specifically, for example, the acquisition unit 7031 determines whether the color information value of each pixel in the image is the same as the color information value indicating approximately orange color, or the color information value indicating approximately blue color, Or, it is compared with other color information values. It is determined that a part of the unmanned aerial vehicle exists at the position of the pixel determined to be the color.

  The acquisition unit 7031 may specify the direction of the unmanned aircraft. Here, the example of the unmanned aerial vehicle shown in FIGS. 2 and 3 will be described. Here, it is assumed that the front of the unmanned aerial vehicle in FIG. 2 is the upper side of the paper surface in FIG. That is, when viewed from the front of the unmanned aircraft, the front is substantially blue, and when viewed from the back of the unmanned aircraft, it is substantially orange.

  11 to 14 show an almost blue color of an unmanned aerial vehicle in an image as an ellipse and an almost orange color as an ellipse with a diagonal line. Here, each color is shown as an abstract ellipse. As a result of analyzing the image of the unmanned aerial vehicle, it is not always possible to acquire image information of the unmanned aircraft as it is due to noise or the like described later.

For example, as shown in FIG. 11, when the left side 1101 is substantially blue (shown by an ellipse in FIG. 11) and the right side 1102 is substantially orange (shown by a slanted ellipse in FIG. 11), The unmanned aerial vehicle is determined to be facing the left with respect to the information processing apparatus that captures an image.

  That is, if the acquisition unit 7031 determines that the X coordinate value of the substantially blue information is smaller than the X coordinate value of the substantially orange information in the image, the unmanned aircraft It is determined that it is facing left. Note that it is not necessary to determine that all the X coordinate values of the substantially blue information are smaller than all the X coordinate values of the substantially orange information due to noise. For example, when a predetermined threshold is provided and the 80% X coordinate value of the substantially blue information is determined to be smaller than the 80% X coordinate value of the substantially orange information, the unmanned aircraft captures information to be imaged. It is good also as a structure which determines with facing a front with respect to a processing apparatus.

  Similarly, as shown in FIG. 12, the right 1202 is substantially blue (shown by an ellipse in FIG. 12) and the left 1201 is substantially orange (see FIG. 12). 12, it is determined that the unmanned aircraft is facing the right with respect to the information processing apparatus that captures an image.

  Further, when the positional relationship between substantially blue and substantially orange in the image is substantially blue (shown by an ellipse in FIG. 13) as shown in FIG. 13, the unmanned aircraft is connected to the information processing apparatus that captures the image. On the other hand, it is determined to face the back.

  Finally, when the position relationship between substantially blue and substantially orange in the image is substantially orange (shown by an ellipse with diagonal lines in FIG. 14) as shown in FIG. It is determined that the information processing apparatus is facing the front.

  In order to determine the direction of the unmanned aerial vehicle, the acquisition unit 7031 acquires information on both the substantially orange color and the substantially blue color of the unmanned aircraft. Specifically, the acquisition unit 7031 searches each pixel of the image, and even if it matches the color that forms part of the unmanned aircraft, the acquisition unit 7031 further continues the search to obtain the substantially orange color of the unmanned aircraft in the image. Get both information in blue.

  In the above description, approximately orange and approximately blue are used. However, any two colors may be used as long as they can be distinguished. For example, it may be a hue that can be distinguished, such as substantially green and substantially blue. Three or more colors may be used. In this case, as compared to comparing the portions drawn with two colors, the direction of the unmanned aircraft can be specified including the drawing portion of the third color, and there is an advantage that the accuracy is further improved.

  Moreover, you may make the application form of a color into another aspect. For example, as shown in FIG. 15, a circle 1502 may be drawn on the side surface of the unmanned aircraft body 1501. By making the airframe 1501 of the unmanned aircraft approximately white and the circle 1502 a specific color, for example, approximately green, it is easy to identify approximately green by the contrast between approximately white and approximately green, and the position of the unmanned aircraft depends on the position. Information can be acquired. Further, as shown in FIG. 16, the mark shape may be doubled. In this case, if the outer circle 1603 is substantially white and the inner circle 1602 is a specific color, the overall color of the fuselage of the unmanned aerial vehicle 1601 can be designed in a different color.

  In FIGS. 15 and 16, the description has been given by using a circle, but the mark may be an arbitrary shape in addition to a polygon such as a square, a rectangle, or a triangle. In addition, if the front, back, right side, and left side of the unmanned aircraft have different colors, the direction of the unmanned aircraft can also be determined based on the color. That is, the storage unit 700 holds information about the color and the direction of the unmanned aircraft in advance in association with the color, and the comparison unit 7031 determines the direction of the unmanned aircraft based on the direction information about the color of the storage unit 700. Can be determined.

  FIG. 17 is an example of the storage unit 700 that stores the relationship between the direction and color of the unmanned aerial vehicle when different colors are given to the four sides of the unmanned aerial vehicle. The color includes information having a certain width as the information of the colors 1 to 4, and the direction of the unmanned aircraft is associated with each color.

  As described above, in the case of being based on the color information, the information on the position of the unmanned aircraft can be acquired by searching for an image. Therefore, there is an advantage that the calculation can be performed comfortably even in a mobile terminal device having limited computer resources.

  Further, when processing is performed by converting to HSV as a color space, since the HSV has a brightness parameter separately from the color, the influence of the flight environment of the unmanned aircraft can be reduced and the image can be recognized with high accuracy. There are advantages.

  In particular, when the size of the unmanned aircraft is within 5 cm, from the viewpoint of safety of the unmanned aircraft, for example, when the distance between the pilot and the unmanned aircraft is 1 m to 5 m, especially 1.5 m to 3 m, more particularly around 2 m, information The size of the unmanned aerial vehicle imaged by the imaging unit of the processing device is small in the amount of information obtained in light of the range that can be identified by the imaging function of the information processing device. Even in such a case, there is an advantage that it is possible to compare and determine functions to be described later in an information processing apparatus with limited calculation resources based on easily identifiable color information. In particular, there are more advantages when calculating in real time.

  If there is noise in the target image, noise may be processed in the acquisition unit 7031 stage. That is, the acquisition unit 7031 may be configured to acquire information while eliminating the influence of noise.

4-3-2. Acquisition Unit 7031: Example of Motion Capture As an example of another method of the acquisition unit 7031, an example of acquiring information related to the position of an unmanned aircraft by motion capture will be described. In motion capture, for example, the difference between a background that moves according to a certain rule and an unmanned aerial vehicle that moves according to a different rule is used. That is, the background operates according to the movement of the information processing apparatus. For example, when the smartphone is moved upward, the background moves downward correspondingly. On the other hand, unmanned aerial vehicles move in response to maneuvers from the transmitter. The movement is different from the background.

  The difference in motion between the background and the unmanned aerial vehicle may be measured based on the difference in images over time. For example, in the image I1 at time t1 and the image I2 at time t2 after t milliseconds, 90% of each pixel of the image I1 is (x−2, y + 3) in the XY coordinate value in the image I2. If it is found that in the remaining 10% that is different from the movement, it is found that an area having the same area as the unmanned aircraft is moving in a certain direction. It is determined that it exists.

4-3-3. Acquisition unit 7031: Information on the position of an example unmanned aircraft based on machine learning may be acquired by machine learning. Machine learning using a set of images containing unmanned aircraft and XY coordinate values indicating the position of the unmanned aircraft as a database to derive rules, and based on the rules, obtain information on the position of the unmanned aircraft May be. In addition, in machine learning, by supplying an image including an unmanned aircraft and information on the unmanned aircraft such as the position or direction of the unmanned aircraft, the unmanned aircraft such as the direction of the unmanned aircraft is separated from the position of the unmanned aircraft described above. You may comprise so that the information regarding an aircraft may be acquired.

4-3-4. Comparison unit 7032
The comparison unit 7032 compares the information related to the position of the unmanned aircraft with the corresponding value of the predetermined value or the predetermined range. For example, when there are a plurality of information related to the position of the unmanned aircraft and there are a plurality of predetermined values or predetermined ranges, the corresponding values are compared. Specifically, for example, if there is an XY coordinate value (3, 8) as information related to the position of the unmanned aircraft, and a predetermined range (X <4, Y <= 8), the X coordinate, Y Each is compared in coordinates. Further, if there is an XY coordinate value (3, 8) as information related to the position of the unmanned aircraft, and there is a predetermined range (X <= 4), the X coordinate 3 is compared with X <= 4. . In this case, the Y coordinate value of 5 is not used. Furthermore, if there is an XY coordinate value (3, 8) as information related to the position of the unmanned aircraft, and there is (X <4, Y> 8) as a predetermined range, the X coordinate and the Y coordinate are respectively compared. .
As another example, the information regarding the position of the unmanned aircraft is (1, 1), (3, 1), (1, 3), (3, 3) square information, and the predetermined value is Let X = 2. In this case, the square and the predetermined value X = 2 can also be compared, and the predetermined value X = 2 exists in a part of the square.

4-3-5. Determination unit 7033
The determination unit 7033 has a function of determining the positional relationship between the position of the unmanned aircraft and a predetermined value or range based on the information compared by the comparison unit 7032. Here, determining the positional relationship determines whether or not the satisfaction is a satisfaction relationship or a non-satisfaction relationship.
When there are a plurality of pieces of information to be compared by the comparison unit 7032, the determination may be determined as being sufficient when all the comparisons are satisfied, or may be determined as being satisfied when some of the comparisons are satisfied. Also good. Whether the determination unit 7033 determines that the satisfaction is satisfied when all the comparisons are satisfied or whether the determination is determined when a part of the comparisons is satisfied may be determined in advance, or a random number or the like may be determined. It may be determined on the spot such as separately calculated and determined by the value.

  For example, if there is an XY coordinate value (3, 8) as the information related to the position of the unmanned aircraft, and there is a predetermined range (X <4, Y <= 8), the X coordinate information 3 is a predetermined range. X coordinate information (X <4) is satisfied, and Y coordinate information 8 is satisfied because Y coordinate information (Y <= 8) in a predetermined range is an inequality sign including an equal sign, both Is satisfied.

  Further, if there is an XY coordinate value (3, 8) as information relating to the position of the unmanned aircraft, and there is (X <= 4) as a predetermined range, the X coordinate 3 is compared with X <= 4, In this case also, it is determined that it is sufficient.

  Furthermore, if there is an XY coordinate value (3, 8) as information related to the position of the unmanned aircraft, and there is (X <4, Y> 8) as a predetermined range, the X coordinate and the Y coordinate are respectively compared. . When the plurality of comparisons are all satisfied, the Y coordinate information 8 and the Y coordinate information (Y> 8) in the predetermined range are not satisfied, so that it is determined that the entire comparison is not satisfied. On the other hand, when a part of a plurality of comparisons is satisfied, the X coordinate information 3 and the predetermined range are satisfied even if the Y coordinate information 8 and the Y coordinate information (Y> 8) of the predetermined range are not satisfied. Since the X coordinate information (X <4) is satisfied, it is determined that it is satisfied as a whole.

  As another example, the information regarding the position of the unmanned aircraft is (1, 1), (3, 1), (1, 3), (3, 3) square information, and the predetermined value is Let X = 2. In this case, the square and the predetermined value X = 2 can also be compared, and since the predetermined value X = 2 exists in a part of the square, it is determined that the square is satisfied.

  As described above, it is determined whether the flight state associated with the predetermined value or the predetermined range has been made by comparing and determining the information related to the position of the unmanned aircraft with the predetermined value or the predetermined range. Can be measured. Using this technology, for example, it is possible to objectively determine whether or not an unmanned aircraft operator has been able to move the unmanned aircraft to a predetermined position in an image captured by the information processing device as a desired flight state. It can be measured. Therefore, for example, in a race in which an unmanned aircraft is moved, it is possible to objectively measure whether or not the unmanned aircraft has moved to a predetermined position.

4-4. Evaluation unit 704
The evaluation unit 704 has a function of evaluating based on part or all of the information obtained by the position comparison determination unit 703.

  The evaluation unit 704 may evaluate based on only part or all of the information obtained by the position comparison / determination unit 703, or part or all of the information obtained by the position comparison / determination unit 703 and other information. You may evaluate after combining information. Other information includes, but is not limited to, the unmanned aircraft usage status and past evaluation scores of the unmanned aircraft such as the total flight time of the unmanned aircraft or the usage status of the last three months. For example, the evaluation may be performed based on various factors such as improving the game performance of the unmanned aircraft described later.

  The evaluation unit 704 may output any information as a result of the evaluation. For example, it may be output as a binary value or a score. The meaning indicated by the binary value may be pass / fail, acquisition success / acquisition failure, and the like. This is, for example, an application for an unmanned aerial vehicle, where a plurality of stages are set and a technique for flying to a predetermined range at each stage is measured. For example, predetermined flight conditions (for example, If the evaluation unit 704 evaluates as pass if the predetermined range is reached in a predetermined time, or the flight is continued for a predetermined time within the predetermined range, and the condition is not satisfied You may comprise so that the evaluation part 704 may evaluate that it is disqualified. In this example, with respect to the time measurement, a measurement technique in the display control unit, which will be described later, is used, and the measured time is considered as one of the elements to be evaluated.

  In the case of the score, different values may be given in relation to a predetermined numerical value or a predetermined range that is a target of the position comparison determination unit 703, or the same value may be used. For example, when it is determined that the position comparison determination unit 703 is satisfied in relation to a certain predetermined value or a predetermined range, it is determined that the position comparison determination unit 703 is satisfied in relation to another predetermined value or a predetermined range. The evaluation unit 704 may be configured to give a higher score than the case. This is because, for example, in order to improve the result of the position comparison determination unit 703 in a certain predetermined value or predetermined range, the result of the position comparison determination unit 703 in another predetermined value or predetermined range is improved. When advanced flight technology is required, the evaluation score can be set according to the level of the flight technology. This in turn helps to increase the incentive for users to aim for advanced flight technology. On the other hand, from the viewpoint of game characteristics, the evaluation unit may be configured to give different points regardless of the flight technique.

  The evaluation unit 704 may be configured to give a negative score (a score to be subtracted). This is because, for example, the evaluation unit is configured to give a negative score such as −10 points, −20 points, etc. in relation to a predetermined numerical value or a predetermined range that is the target of the position comparison determination unit 703. To do. For example, in the case of a dangerous area flight such as a flight of an unmanned aerial vehicle, the negative score has significance such as teaching a risk or alerting the user, and is useful for improving a user's flight technique. As a flight in a dangerous area, for example, a certain object, a living thing, a flight prohibited area, or the like can be cited as a background image.

Further, the evaluation unit 704 may have a function of adding together a plurality of evaluation results based on part or all of the information obtained by the position comparison determination unit 703. As described above, since subtraction can also be performed, the score of the summation result can be a positive number or a negative number. Further, the score may be an integer, a rational number, a decimal point, or the like.

For example, when the position of the unmanned aircraft is within a predetermined XY coordinate range for a predetermined period of time, it may be evaluated as passing. In addition, when the position of the unmanned aircraft is within a predetermined XY coordinate range for a predetermined period of time, an evaluation may be made such that a score is given.
In addition, a plurality of stages may be set, and in each stage, a score obtained by acquiring a mark given to a predetermined location in the image (an unmanned aircraft touches the mark) may be an evaluation target, The evaluation target may be that each stage is cleared when the score reaches a certain level.

  The evaluated result may be stored in the storage unit 700. In addition, past evaluation results may also be stored in the storage unit. For example, FIG. 18 is an example of a database of information relating to the flight of unmanned aircraft relating to the user. In FIG. 18, the number of stars is recorded. This corresponds to the number of star marks acquired at the stage, and indicates that a good evaluation was obtained at the stage. Note that a score may be recorded instead of the number of stars. The evaluation may be performed in association with date / time information, flight time information, and the like.

  Alternatively, the position of the unmanned aircraft may be stored at regular time intervals, and the amount of change may be used as a position element for evaluation. In other words, it is important to refrain from rapid movement in order to stably control the unmanned aircraft. This can be done by calculating the change in travel distance per unit time, that is, the speed, by comparing the position of the unmanned aircraft every fixed time, or by calculating the change in speed per unit time, that is, the acceleration. May be.

  FIG. 19 is an example of a table in which the position of the unmanned aircraft is acquired with XY coordinate values at regular time intervals, the speed is calculated from the change amount of the position, the acceleration is calculated from the change amount of the speed, and stored. . Information on the position Xn (1 <= n) of the unmanned aerial vehicle is acquired at a time Tn (1 <= n) every fixed time t, and the speed and acceleration are obtained by a known method based on the position information. Is calculated. The acceleration may be compared with a predetermined value and used as an evaluation element. For example, the acceleration may be compared with a plurality of predetermined values, and higher evaluation may be given when moving at a lower acceleration than when moving at a higher acceleration.

  The score may be managed for each user, for each user ID, for each group to which the user belongs, and so on.

4-5. Display control unit 705
The display control unit 705 has a function of causing the display unit of the information processing device to perform various displays. The display control unit 705 may display an image obtained by imaging an unmanned aerial vehicle, or may superimpose an image obtained by imaging the unmanned aircraft and display a predetermined value or a predetermined range as an image. In addition, the display control unit 705 may display an image associated with a predetermined value or a predetermined range by superimposing the image on the unmanned aircraft. Examples of the image associated with the predetermined value or the predetermined range include, for example, an image substantially equal to the predetermined range, an image that is slightly larger than the predetermined range, or a shape of the predetermined range. It may be an image of a related graphic. The display control unit 705 may display a background image on the display unit. The background image may be a real landscape image or an imaginary image, for example. Also, an image such as an animation may be displayed, or an image related to a game may be displayed. The user can get a feeling of flying in a real landscape image or an imaginary image while flying the unmanned aircraft, and can fly the unmanned aircraft more happily.

  Further, the display control unit 705 may cause the display unit to display information based on a part or all of the information obtained by the position acquisition determination unit 703, or a part of the information obtained by the evaluation unit 704 or Information based on the whole may be displayed on the display unit. In addition, based on information based on part or all of the information obtained by the position acquisition determination unit 703 or information based on part or all of the information obtained by the evaluation unit, the execution result of the application described below is obtained. You may display on a display part. Examples of display related to the application include part or all of the application title screen, menu screen, tutorial screen, game screen, evaluation screen, personal data screen, setting screen, and other screens in the necessary order. You may display along. For example, FIG. 20 etc. are mentioned as a game screen which acquires the above-mentioned score. In FIG. 20, a star mark that is acquired by overlapping with the position of the unmanned aircraft is displayed as one screen of the game. The star mark is acquired, and points are added. The score is 300 for the star 2002 and 2000 for the star 2003. The score is displayed on a star mark so that the score when it is acquired can be easily understood.

The display control unit 705 displays information based on part or all of the information obtained by the position acquisition determination unit 703, so that the driver of the unmanned aircraft moves the unmanned aircraft to a predetermined position as a desired flight state. Can understand objectively whether or not
Further, the display control unit 705 may have a measurement function for measuring time in association with a part of or all of the information obtained by the display content or the position acquisition determination unit 703. The display control unit 705 may measure the time based on the absolute time such as recording the start time and the end time and measuring the time based on the difference between the records when measuring the time. Then, a relative elapsed time such as measuring the time from the start point to the end point may be measured.

  The measurement function in the display control unit 705 can also measure the time from a predetermined condition until the positional relationship determination unit 703 determines the positional relationship, or after the positional acquisition determination unit 703 determines the positional relationship, Measurement can also be started. Further, for example, the predetermined conditions include that the application has started, that the countdown indicated in the application has ended, that the position of the unmanned aircraft has started to move, and that the unmanned aircraft has been determined by the position comparison determination unit. The predetermined condition may be that they are in the positional relationship. The measurement function can also be used as various measurement functions used in an application or a game.

  Information related to individuals may be displayed on the personal data screen. Information about the individual may include information such as name, ID, nickname, image, maximum score in each stage, number of challenges for each stage, level, total score for all stages, total flight time, and the like. These pieces of information are stored in, for example, a table as shown in FIG.

  On the setting screen, various settings regarding the application can be displayed and changed. This may be setting of data related to an individual, setting, changing, resetting, etc. of a game level described later.

  The game screen may be any type of game. In other words, the game may be any game that can be enjoyed while the user views the display screen of the information processing apparatus with the flight state of the unmanned aircraft in view when operating the unmanned aircraft. Note that it is even better if the game improves the user's unmanned aircraft flight technology.

4-6. Configuration of Information Processing Apparatus The information processing apparatus in the embodiment of the present invention is configured by the various information processing apparatuses described in 1 above. The mobile terminal device may have all the functions of the above-described units. By reducing the processing load of image processing and the like, an embodiment of the present invention can be implemented even in a portable terminal device with low processing capability.

  Further, part or all of the display control unit may be executed on the head mounted display. In this case, there is an advantage that the operator can handle the transmitter without worrying about the imaging function of the information processing apparatus provided in the transmitter. In addition, when the display control unit presents a game, it is possible to concentrate on the game with a more immersive feeling.

  Further, all the information processing functions according to the present invention may be realized on the head mounted display. In this case, the head mounted display has an imaging function, and the above-described processes such as image processing may be performed on the head mounted display.

  In one embodiment of the present invention, the mobile terminal device may have some functions, and other devices such as a server device and a cloud may have the remaining functions. For example, an image including an unmanned aircraft in a field of view is acquired by a mobile terminal device, the image is transferred to another device such as a cloud or a server device via a communication IF, and image processing and predetermined processing are performed by the other device. It is also possible to adopt a configuration in which comparison or determination with the value or range is performed, and the result is transferred to the mobile terminal device. In this case, the configuration may be such that a third party owns or manages the other device, performs image processing and comparison with a predetermined value or range, and only the result is transferred to the mobile terminal device. Needless to say, the server device may realize each function by cooperation of a plurality of server devices.

5). Hereinafter, an embodiment of the application according to the present invention will be described. Each application may include steps as shown in the flowchart of FIG.

  For example, the display control unit may display the title screen of the application on the display unit (S1). Here, since it is a flight of an unmanned aerial vehicle, an alert regarding safety, basic usage, and the like may be displayed.

  In addition, the display control unit may display a login screen on the display unit (S2). The display control unit may acquire the user ID via the login screen. Further, the display control unit may acquire a password associated with the user ID. The display control unit can identify the user by authenticating using the user ID and password provided in the storage unit. Data and / or settings associated with the user may also be invoked. Information related to the user ID and password may be stored in an information processing apparatus such as a remote server and communicated therewith for authentication.

  Next, the display control unit may display a menu screen on the display unit (S3). Here, an example in which a measurement mode having actual game characteristics, a mode in which flight is enjoyed without measurement for an unmanned aerial vehicle, and a page for information on the unmanned aircraft may be disclosed. In addition, the mode to enjoy flying without taking measurements for unmanned aerial vehicles may be able to be used after having achieved a certain result in the measurement mode and equipped with technology that can fly unmanned aircraft with some degree of stability. .

  In addition, the display control unit may explain an example of the tutorial related to the basic operation method (S4). Through the tutorial, unmanned aerial pilots can learn basic skills. Further, the display control unit may display a game (S5). Hereinafter, an example of an unmanned aircraft measurement mode or an unmanned aircraft game will be described with reference to flowcharts.

5-1. Embodiment 1
Next, an embodiment of an application relating to the present invention will be described with reference to the flowchart of FIG. FIG. 23 is an embodiment for measuring the state in which an unmanned aerial vehicle flies for a certain period of time at a position associated with a predetermined range.

  First, the display control unit displays an image acquired from the imaging device on the display unit (S1). The image is appropriately updated as time passes.

  Next, the display control unit displays an image corresponding to a predetermined range set in advance in the display unit (S2).

  Next, the position acquisition determination unit determines whether there is an unmanned aircraft within a predetermined range in the image at regular time intervals (S3). The fixed time may be a fine time interval such as 10 milliseconds, or a coarse time interval such as 0.5 seconds.

  When the position acquisition determination unit determines that there is an unmanned aerial vehicle within a predetermined range, a predetermined time is posted (S4). The predetermined time is a target time at which the unmanned aircraft is expected to fly continuously within a predetermined range. At this time, the display control unit may display some character string related to the time. For example, when the predetermined time is 10 seconds, characters such as “Hover over a white display range for 10 seconds” may be used.

  Next, as long as the position acquisition determination unit continues to determine that the unmanned aircraft is flying within a predetermined range, the display control unit measures and displays the remaining time of the predetermined time (S5). At this time, the display control unit may display a character string corresponding to the elapsed time. For example, the content may be such as “Successful hovering for 5 seconds! Continue hovering for the remaining 5 seconds!” Or “Do your best for the remaining 3 seconds!”

  The character string displayed by the display control unit may be advice for the user. For example, the predetermined range is a double rectangle of 3101 and 3102 as shown in FIG. 31, the outer 3101 range is displayed by the display control unit, and the inner 3102 range is not displayed. Assuming that And if there is an unmanned aerial vehicle 3103 between the outer side 3001 and the inner side 3102, it is more likely to come out of the outer side 3101 than when inside the inner side 3102. If it is, an advice to move to the inner side 3102 may be displayed on the display unit, such as “It is better to rise further”.

  When the display control unit determines that the remaining time is 0, if the position acquisition determination unit determines that there is an unmanned aircraft within a predetermined range, the display control unit displays that the target time has been successfully hovered. (S6). For example, “10 seconds hovering success!” Or the like may be displayed on the display unit.

  On the other hand, if the position acquisition determination unit determines that there is no unmanned aircraft within the predetermined range before the display control unit determines that the remaining time is 0, the display control unit indicates that the hovering of the target time has failed. Display (S7). For example, the display is “Sorry, hovering failure for 10 seconds!”. At this time, the remaining time may be displayed. For example, “2 seconds remaining, let's do our best next time!” May be displayed on the display unit.

  In the above description, the hovering is performed for a predetermined target time, but the elapsed time may be measured. That is, the time measurement starts from the time when the unmanned aircraft enters the predetermined range. Then, the time during which the flight can continue within a predetermined range may be recorded as the maximum hovering time.

  In this embodiment, since it is measured whether or not the unmanned aircraft has been able to fly at a position related to a predetermined range for a certain period of time, it is possible to objectively measure the flight technique of the driver's unmanned aircraft.

  In addition, flying for a certain period of time to a position associated with a predetermined value or a predetermined range means that the driver of the unmanned aircraft moves the unmanned aircraft to a desired position and a position related to the desired position. In addition, it also relates to measuring techniques for safely flying unmanned aerial vehicles because it means that the aircraft can fly continuously for a desired time.

  The predetermined range of the above embodiment may be a predetermined value. The predetermined value or the predetermined range may be set in the application, or the predetermined value or the predetermined range may be set based on information input by the user. The predetermined value or the predetermined range may be set by a numerical value input by the user on the application setting screen.

  When the predetermined range is set to about 1.5 to 3 times larger than the unmanned aircraft size when the unmanned aircraft is imaged by the information processing device, the unmanned aircraft Since it is easy to fly within the range, even beginners of unmanned aerial vehicles can easily sense the sense of success of hovering.

  Further, when a predetermined range is set to about 1.5 times or less than the unmanned aircraft size when the unmanned aircraft is imaged by the information processing apparatus, the unmanned aircraft is narrowed to a predetermined range. Since it is difficult to fly within this range, even an unmanned aerial vehicle operator can feel a sense of achievement of hovering.

  Further, the predetermined range may be set to a narrow range after the predetermined range is set to a wide range according to a predetermined condition. In this case, users of unmanned aerial vehicles first experience unmanned aerial vehicle maneuvering over a wide predetermined range, and as the level of technology improves, they can be easily realized within the initial wide predetermined range. Even after you no longer remember, you can experience the hovering technology within a narrow range, and there is an advantage that you can challenge to improve the technology. In this case, the predetermined condition can be set based on various conditions such as the total flight time, the start time from the time when the application is started, and the stage difference described later.

  In addition, the display control unit compares the area of the predetermined range or a numerical value related thereto (for example, the length of one side in the predetermined range of the rectangle, the length of the radius of the predetermined range of the circle, etc.) numerically. , May be displayed. It is possible to objectively compare that the predetermined range is narrowed numerically. When the numerical value is displayed, the operator can objectively understand that the target value is narrowed.

  In addition, if the fixed time is set short, it is easy to fly the unmanned aircraft within a predetermined value or within a predetermined range within the short fixed time. There is an advantage that you can feel the sensation easily.

  In addition, if the fixed time is set long, it is difficult to fly the unmanned aircraft within a predetermined value or a predetermined range within the long fixed time. There is an advantage that you can feel the skill.

  Further, the predetermined time may be set longer after the predetermined time is set shorter according to a predetermined condition. In this case, unmanned aerial aircraft users initially experience hovering in a short period of time, and as the level of technology improves, it can be easily realized in the initial short period of time, and then hovering for a long period of time. There is an advantage that you can experience technology and challenge to improve technology. In this case, the predetermined condition can be set based on various conditions such as the total flight time, the start time from the time when the application is started, and the stage difference described later.

  Also, the measurement time may be compared and displayed numerically. It is possible to compare numerically that the measurement time is long. When the numerical value is displayed, the operator can objectively understand that the target value has increased.

  In addition, an application that can improve various measurement techniques for improving the hovering technique by combining a certain period of hovering with a predetermined value or a range of a predetermined range may be used.

  When the imaging function provided in the portable terminal device is used, an individual can easily measure the flight technology of an unmanned aerial vehicle. Therefore, individuals can easily measure their skills.

5-2. Embodiment 2
Next, an embodiment of an application relating to the present invention will be described with reference to the flowchart of FIG. In FIG. 24, an embodiment is shown in which an unmanned aerial vehicle measures a technology that reaches a predetermined range within a requested time.

  The image acquisition unit displays the image acquired from the imaging device on the display unit (S1).

  The display control unit displays a predetermined range on the display unit as the target hovering position (S2). For example, the target hovering position is displayed on the display unit, and a description of how many seconds within the hovering is requested is displayed on the display unit. For example, it is displayed on the display unit such as “Aim at hovering within a thick frame within 20 seconds”.

  Next, the display control unit displays a countdown before the start of hovering on the display unit (S3). The countdown may be displayed on the display unit. For example, “5 seconds”, “4 seconds”, “3 seconds”, “2 seconds”, “1 second”, “GO”, and the like.

  Next, the display control unit counts down for a predetermined time (first countdown) until the target hovering is performed within a predetermined range, and displays the first countdown on the display unit (S4). ). For example, “20 seconds” and “19 seconds”.

  The position acquisition determination unit determines whether or not the unmanned aircraft is flying within a predetermined range at regular time intervals (S5). The fixed time may be a time interval such as 10 milliseconds to 1 second.

  The display control unit may display a display that appeals to the user when the first countdown approaches zero. For example, a thick frame indicating a predetermined range may be changed in color such as a red frame, blinked, or the entire screen may be lit white.

  If the position acquisition determination unit determines that the unmanned aircraft is flying within a predetermined range, the display control unit starts another predetermined time countdown (second countdown), and the character string of the description At the same time, the second countdown is displayed on the display unit (S6). For example, a message such as “Achieving the target! Do your best to hover for another 3 seconds!” Is displayed on the display unit. Note that while the position acquisition determination unit determines that the unmanned aircraft is flying within a predetermined range, the display control unit stops the first countdown.

  When the position acquisition determination unit determines that the unmanned aircraft is flying within the predetermined range until the display control unit finishes the second countdown, the display control unit displays “Successful hovering!” (S7).

  If the position acquisition determination unit determines that the unmanned aircraft is not flying within a predetermined range before the display control unit finishes the second countdown, the display control unit At this time, the display control unit may display on the display unit, for example, “Go to a predetermined range!”.

If the unmanned aircraft does not enter the predetermined range even when the countdown of the display control unit becomes 0, the display control unit displays “Game failure!” On the display unit (S9).
In the second embodiment, the predetermined range and the requested time can also be set variously in the same manner as the predetermined value or range and the predetermined time in the first embodiment.
The second embodiment also has the same effect as the first embodiment.

5-3. Embodiment 3
Next, an embodiment of an application related to the present invention will be described with reference to the flowchart of FIG. The game shown in FIG. 25 is a game in which a certain area is displayed in the display unit, and a score is obtained by moving an unmanned aircraft outside the area within a certain time. Instead of acquiring the points, if the unmanned aircraft does not move from the area within a certain time, a point may be deducted.

  First, the image acquisition unit displays the image acquired from the imaging device on the display unit (S1).

  Next, the display control unit acquires the position of the unmanned aircraft (S2).

  Next, the display control unit sets a certain area including the position of the unmanned aircraft as a predetermined range (S3). Here, the display control unit may indicate an instruction to the user. For example, “Escape from the range of the white line within 20 seconds!”. Note that the predetermined range is preferably displayed. This is to allow the user to visually understand in which range the unmanned aircraft is allowed to fly and the points cannot be obtained or deducted. However, it is also possible to display only the escape from the current location without displaying the predetermined range.

  Next, the display control unit starts counting down from a predetermined value (S4). At this time, the display control unit may display the countdown on the display unit. For example, “Escape from the white line in 18 seconds!”

  The display control unit may change the image in the display unit when the countdown approaches zero. As a change mode, for example, a certain region in the display unit may be surrounded by a line, or the color of the certain region in the display unit may be changed. In addition, the area may be enclosed in a form in which the center is more emphasized, or may be displayed in a color in which the center is more emphasized.

  When the position acquisition determination unit determines that the unmanned aircraft has moved out of the predetermined range, the position acquisition determination unit records the related time (S5). The display control unit may record the time of movement outside a predetermined range as the related time. Further, the display control unit may record the elapsed time from the start time of the game to the time when the unmanned aircraft moves out of a predetermined range as the related time. In addition, the display control unit may display that it has moved from a predetermined range. For example, the display control unit may display “Escape Success!” On the display unit. Further, the display control unit updates the information when the related time is shorter than the previous related time in the stage.

  If the unmanned aircraft does not move out of the predetermined range even when the predetermined countdown reaches 0, the display control unit displays this (S6). For example, the display control unit may display “Game failure!” On the display unit.

  Next, with reference to FIG. 26, a game in which the unmanned aircraft obtains a predetermined mark by acquiring a predetermined mark will be described.

  First, the image display unit displays a game title (S1). Moreover, you may display the simple description of a game. For example, “Get more stars and aim for a higher score!”.

  Next, the image display unit presents user data (S2). Here, the number of challenges for each stage, the highest score for each stage, the total flight time, and the like may be displayed. Note that the score of a stage that has not yet been executed may be displayed as 0 points or may not be displayed. Each stage may be set with a minimum score for clearing that stage. That is, the stage is cleared only after obtaining the lowest score. The order may be set for the stages. In that case, the user can challenge the next stage in which the order is determined in advance by clearing the stage.

  Next, when the actual action of the game starts, the image acquisition unit acquires an image acquired from the imaging unit including the unmanned aircraft in the field of view, and the display control unit displays the image on the display unit (S3).

  The display control unit displays the mark at a position in the image (S4). The mark is handled as a predetermined range in the display control unit. Here, the storage unit may have a table in which the mark type and the position to be presented in the image are defined. For example, you may have a database like FIG. The display control unit designates each mark and displays it in the image. The position in the displayed image may be presented at a presentation position associated with the mark in the storage unit as shown in FIG. 27, or may be changed according to the situation. For example, the display of a position far away and a position close to the current unmanned aircraft position may be repeated in order. In addition, the timing of presenting the mark may be determined in advance, or the timing may be determined based on information obtained from the previous mark. For example, the next mark may be displayed 2 seconds after the previous mark is acquired. A plurality of marks may be displayed simultaneously. Further, the display control unit may determine a mark to be presented according to a random number or the like.

  The position acquisition determination unit determines whether or not the mark falls within a predetermined range based on the image processed information (S5).

  When the position acquisition determination unit determines that the unmanned aircraft is within the predetermined range, the acquisition of the related mark is recorded (S6).

  The position acquisition determination unit erases the display of the mark (S7).

  When the display control unit determines that a certain period has elapsed since the deletion of the previous mark, the display control unit displays a new mark on the display unit (S8).

  The display control unit ends the game when a certain time has elapsed from the start of the game (S9).

  The display control unit adds up the marks obtained in the stage and records them in the storage unit (S10). If the score is higher than the score of the past mark obtained in the stage or the highest score of the past mark, the information on the highest score in the stage is updated. If the highest score is higher than the clear score in the stage, and the next stage cannot be executed, the next stage is changed to be executed. In addition, the flight time in the stage is added to the total flight time.

  In the third embodiment, the game performance is further improved through the acquisition of points. As a result, the driver of the unmanned aerial vehicle can enjoy the unmanned aircraft more and improve the maneuvering technology of the unmanned aerial vehicle.

5-4. Embodiment 4
Next, an embodiment of an application related to the present invention will be described with reference to the flowchart of FIG. FIG. 28 illustrates a game in which an unmanned aircraft virtually suffers damage in relation to a predetermined range. For the virtual damage, the number of times related to the predetermined range may be determined, or the number of points associated with the predetermined range may be deducted. The number of times determined to be within the predetermined range may be determined in advance. In this game, points may be scored as in the above example. In this game, differences from the previous game will be mainly described.

  When the actual action of the game starts, the image acquisition unit acquires an image acquired from the imaging unit including the unmanned aircraft, and the display control unit displays the image on the display unit (S1).

  The display control unit starts measuring time from the start of the game (S2). The time may be displayed on the display unit.

  The display control unit displays the obstacle at a position in the image (S3). This obstacle may be associated with a predetermined range. For example, the outer edge of the obstacle may coincide with a predetermined range. Obstacle images may be tailored to the world view of each stage. For example, in a stage where the interior of the building is a world view, it may be present inside the building. Specifically, for example, if it is a view of the world inside a general house, it may be related to buildings such as ceilings, walls, floors, stairs, doors, windows, chests, figurines, TVs, refrigerators. It may be a home appliance such as a microwave oven. Even inside a building, a desk, a chair, a whiteboard, a projector, or the like may be used as long as it is an image of a company office, meeting room, or the like. In addition, as a commercial facility, in a stage such as a department store, a theme park, or the like, what is generally assumed in each stage may be set as an obstacle.

  In addition to artifacts, natural environments such as the sea, mountains, forests, parks, etc. are used as the stage. For example, plants, animals, rocks, etc. It may be set.

  Also, apart from everyday life, historical buildings, world heritage sites, space, desert, deep sea, air, etc. may be used as stages, and what may be present there may be obstacles. The world can be a stage.

  The outer edge of the obstacle may not coincide with the predetermined range. For example, a predetermined range may be set slightly inside from the outer edge of the obstacle. For example, in the case of an obstacle due to an image of a tree or a forest, there may be a case where the unmanned aircraft is not damaged just by touching the unmanned aircraft a little.

  The obstacle may move. For example, an unmanned aerial vehicle may be moved relatively by moving an obstacle as a background. For example, in FIG. 29, an image of an underground tunnel is used as the background. The tunnel 2901 moves in the direction of the arrow 2903 (left direction) as viewed from the drawing sheet, and the unmanned aircraft 2902 moves in the opposite direction (right direction).

  The obstacle may be an enemy in the stage. Examples include monsters, monsters, battleships, fighters, and tanks. The obstacle may be a virtual attack means from the enemy. For example, it may be a bullet from a tank, a battleship, a fighter plane, or a beam attack from a monster, a monster, a battleship, or the like. An example is shown in FIG. In FIG. 30, an enemy 3002 appears with respect to the unmanned aircraft 3001 to be operated. The enemy 3002 may be matched with a predetermined range, and the unmanned aircraft may acquire predetermined damage when it comes into contact with the enemy 3002. The enemy 3002 can fire a virtual attack means 3003. A predetermined range is also set in 3003, which moves at a speed simulating a constant bullet. When the unmanned aerial vehicle 3001 comes into contact with the attack means 3003, the 3001 can be configured to deduct a predetermined score.

  A display control part performs the display which moves the obstruction which is a background (S4).

  The position acquisition determination unit determines a relationship between the unmanned aircraft and a predetermined range associated with the obstacle based on the image-processed information (S5).

  When the position acquisition determination unit determines that the unmanned aircraft is within the predetermined range, it records this (S6).

  The display control unit determines whether or not the number of times determined to be the predetermined range associated with the obstacle is greater than the predetermined number of times (S7).

  When the number of times that the display control unit is determined to be within the predetermined range associated with the obstacle is less than the predetermined number of times, the display control unit becomes S3 (S8).

  When the number of times determined to be within the predetermined range associated with the obstacle is greater than the predetermined number, the display control unit ends the game (S9).

  After the game is over, the measured time is displayed (S10).

  The display control unit records the time in the storage unit (S11). If the time is longer than the past time obtained in the stage, the information on the maximum time in the stage is updated. If the maximum time is longer than the clear time in the stage and the next stage cannot be executed, the next stage is changed to be executed. In addition, the flight time in the stage is added to the total flight time.

  In this example, as in the previous example, the score may be acquired by acquiring the mark. Further, the unmanned aircraft to be controlled in response to the enemy being provided with attack means may also be provided with virtual attack means. The virtual attack means may be configured to be launched in response to the operation means in the unmanned aircraft transmitter. Further, the attack means may be configured to be acquired as an item corresponding to the acquisition of the mark.

  In the fourth embodiment, the range of game characteristics is further improved by means of deductions or the appearance of enemies than in the third embodiment. As a result, the driver of the unmanned aerial vehicle can more enjoy the operation of the unmanned aerial vehicle, and the maneuvering technology of the unmanned aerial vehicle can be improved.

  The processes and procedures described in the present specification can be realized not only by those explicitly described in the embodiment but also by software, hardware, or a combination thereof. Further, the processes and procedures described in this specification can be implemented as a computer program and executed by various computers.

Claims (25)

An imaging unit for imaging an unmanned aerial vehicle in flight based on an operation by a transmitter;
An image processing unit that performs image processing on part or all of an image based on the imaging unit;
A position comparison / determination unit that compares a predetermined range with information related to the position of the unmanned aircraft in the image based on part or all of the information obtained from the image processing unit;
A display unit for displaying the imaged unmanned aircraft;
An evaluation unit that evaluates the flight state of the unmanned aircraft based on part or all of the information obtained by the position comparison determination unit;
An information processing apparatus different from the transmitter, which is capable of objectively measuring a flight technique of the unmanned aircraft by the operator of the transmitter. The display unit displays the predetermined range and a remaining time expected to hover within the predetermined range;
The evaluation unit evaluates the success or failure of the hovering of the time expected to be hovered by the unmanned aircraft based on part or all of the information obtained by the position comparison determination unit.
The information processing apparatus according to claim 1. The time when the unmanned aircraft has continued to fly within the predetermined range can be recorded as a hovering time.
The information processing apparatus according to claim 1 or 2. The display unit displays a first rectangle;
The predetermined range is a second rectangle set inside the first rectangle;
The display unit displays a character string when the unmanned aircraft is located between the first rectangle and the second rectangle.
The information processing apparatus according to any one of claims 1 to 3. In the case where the first acceleration is lower than the second acceleration, the evaluation unit is higher than the evaluation in the case where the unmanned aircraft is the first acceleration and the unmanned aircraft is the second acceleration. Give a rating,
The information processing apparatus according to any one of claims 1 to 4. The image processing unit includes a process of converting to the HSV method,
The image processing unit obtains information related to the position of the unmanned aircraft using the converted hue in the HSV method.
The information processing apparatus according to any one of claims 1 to 5. The information processing apparatus includes a CPU.
The information processing apparatus according to claim 1. The information processing device is a mobile terminal device.
The information processing apparatus according to any one of claims 1 to 7. An information processing device different from the transmitter capable of objectively measuring the flight technology of an unmanned aircraft by a transmitter operator,
Imaging means for imaging the unmanned aerial vehicle in flight based on an operation by the transmitter;
Image processing means for image processing part or all of an image based on said image pickup means,
Said image processing based on a part of or all of information obtained from the means, the information relating to the position of the unmanned aerial vehicle, the positional relationship determining means for determining the positional relationship by comparing a predetermined range, and
Display means for displaying the imaged unmanned aircraft,
Evaluation means for evaluating the flight state of the unmanned aircraft based on part or all of the information obtained by the positional relationship determination means;
Computer program to function as. The display means displays the predetermined range and a remaining time expected to hover within the predetermined range;
The evaluation means evaluates the success or failure of the hovering time expected to be hovered by the unmanned aircraft based on part or all of the information obtained by the positional relationship determination means .
The computer program according to claim 9. Recording means capable of recording the information processing device as a hovering time when the unmanned aircraft continues to fly within the predetermined range;
The computer program according to claim 9 or 10 for functioning as: The display means displays a first rectangle;
The predetermined range is a second rectangle set inside the first rectangle;
The display means displays a character string when the unmanned aircraft is located between the first rectangle and the second rectangle.
The computer program according to any one of claims 9 to 11. In the case where the first acceleration is lower than the second acceleration, the evaluation means is higher than the evaluation in the case where the unmanned aircraft is the first acceleration and the unmanned aircraft is the second acceleration. Give a rating,
The computer program according to any one of claims 9 to 12. The image processing means includes a process of converting to the HSV system,
The image processing means acquires information related to the position of the unmanned aircraft using the hue in the converted HSV method.
The computer program according to any one of claims 9 to 13. The information processing apparatus includes a CPU.
The computer program according to any one of claims 9 to 14. The information processing device is a mobile terminal device.
The computer program according to any one of claims 9 to 15. An information processing device different from the transmitter capable of objectively measuring the flight technology of an unmanned aircraft by a transmitter operator,
An imaging step of imaging the unmanned aerial vehicle in flight based on an operation by the transmitter;
An image processing step of performing image processing on part or all of the image based on the imaging step ;
Based on a part or all of the information obtained from the image processing step, the positional relationship determination step of determining the positional relationship by comparing the information related to the position of the unmanned aircraft with a predetermined range;
A display step for displaying the imaged unmanned aerial vehicle;
An evaluation step for evaluating the flight state of the unmanned aerial vehicle based on part or all of the information obtained in the positional relationship determination step ;
How to run. The display step includes a step of displaying the predetermined range and a remaining time expected to hover within the predetermined range;
The evaluation step includes a step of evaluating success or failure of hovering at a time expected to be hovered by the unmanned aircraft based on part or all of the information obtained in the positional relationship determination step .
The method of claim 17. The information processing apparatus can record the time that the unmanned aircraft has continued to fly within the predetermined range as the hovering time.
The method according to claim 17 or 18. The displaying step includes displaying a first rectangle;
The predetermined range is a second rectangle set inside the first rectangle;
The display step includes a step of displaying a character string when the unmanned aerial vehicle is located between the first rectangle and the second rectangle.
20. A method according to any one of claims 17-19. In the case where the first acceleration is lower than the second acceleration, the evaluation step is higher than the evaluation in the case where the unmanned aircraft is the first acceleration and the unmanned aircraft is the second acceleration. Including the step of giving a rating,
21. A method according to any one of claims 17 to 20. The image processing step includes a process of converting to the HSV method,
The image processing step includes a step of obtaining information on the position of the unmanned aircraft using the hue in the converted HSV method.
The method according to any one of claims 17 to 21. The information processing apparatus includes a CPU.
23. A method according to any one of claims 17-22. The information processing device is a mobile terminal device.
24. A method according to any one of claims 17 to 23.   A computer program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 8.

Images