JP6661143B2 - Moving object, control system, control method, and program - Google Patents

Description

  The present invention relates to a moving object, a control system, a control method, and a program.

  As a mechanism for controlling a mobile object called a drone, for example, Patent Document 1 discloses a system in which a control command is wirelessly transmitted from a tablet operated by a user to the mobile object.

JP-A-2006-94188

  The mechanism described in Patent Document 1 is to remotely control a moving body in real time by an operation of an operator, but in the future, the moving body will autonomously operate without a human being instructing in real time. It is desired to control its own operation. In addition, since a moving object is a device that performs operations such as flying even in a human living space, it is desirable for a person around the moving object to easily understand how the moving object operates.

  An object of the present invention is to provide a technology that enables a moving body to operate without remote control of the moving body in real time by an operation of an operator and that makes it easy for a human to understand what kind of operation the moving body performs. To provide.

Means to solve the problem

According to the present invention, a plurality of pieces of operation information each indicating a condition and an operation to be performed when the condition is satisfied by at least one of a character, a symbol, and a graphic that can be interpreted by a human being, wherein the operation information indicates an operation performed by the moving object. The display object displayed in a different color , and an imaging unit that captures an object to be subjected to the operation, and analyzes a captured image in which the display object and the object are captured, and analyzes the shape, color, or An operation execution unit that determines whether a size satisfies a condition indicated by the operation information , and executes an operation indicated by the operation information on the object when the condition is satisfied; When the condition indicated by the operation information displayed in the first color is satisfied, the execution unit adds the operation information displayed in the second color to the operation information. Performs the operation indicated by the operation information displayed in the first color, regardless of whether or not the condition indicated by the operation information displayed in the first color is satisfied. If the condition indicated by the operation information displayed in the second color is satisfied, the operation indicated by the operation information displayed in the second color is performed. Provide moving objects.

The condition is displayed by the character, and the operation execution unit stores the handwriting of the character recognized from the display object in the captured image when the condition indicated by the operation information is satisfied. When the handwriting matches the handwriting, the operation indicated by the operation information may be executed.

In addition, the present invention provides a plurality of operation information indicating an operation performed by a mobile object, which indicates a condition and an operation to be performed when the condition is satisfied by at least one of a character, a symbol, and a graphic that can be interpreted by a human. A display object in which the motion information is displayed in different colors , and an imaging unit that captures an object to be subjected to the operation, and a captured image in which the display object and the object are captured are analyzed, and the shape of the object, An operation execution unit that determines whether a color or a size satisfies a condition indicated by the operation information , and executes an operation indicated by the operation information on the object when the condition is satisfied, The operation execution unit is configured to, when a condition indicated by the operation information displayed in the first color is satisfied, perform the operation displayed in the second color. The operation indicated by the operation information displayed in the first color is executed regardless of whether or not the condition indicated by the information is satisfied, and the condition indicated by the operation information displayed in the first color is satisfied. If the condition indicated by the operation information displayed in the second color is satisfied, the operation indicated by the operation information displayed in the second color is performed. Provide a control system.

In addition, the present invention provides a plurality of operation information indicating an operation performed by a mobile object, which indicates a condition and an operation to be performed when the condition is satisfied by at least one of a character, a symbol, and a graphic that can be interpreted by a human. A display object in which motion information is displayed in different colors , and an imaging step of imaging an object to be subjected to the operation, and analyzing a captured image in which the display object and the object are imaged , the shape of the object, Determining whether the color or the size satisfies the condition indicated by the operation information , and performing an operation indicated by the operation information on the object when the condition is satisfied, In the operation execution step, when a condition indicated by the operation information displayed in the first color is satisfied, Regardless of whether or not the condition indicated by the operation information displayed in the second color is satisfied, the operation indicated by the operation information displayed in the first color is executed, and the operation is performed in the first color. If the condition indicated by the operation information is not satisfied and the condition indicated by the operation information displayed in the second color is satisfied, the operation indicated by the operation information displayed in the second color Is provided.

Further, the present invention provides a computer provided with a mobile object, provided that the operation information indicating the operation performed by the mobile object is at least one of a character, a symbol, and a graphic that can be interpreted by a human, and the condition and the condition are satisfied. Analyzing a display object in which a plurality of pieces of operation information indicating respective operations to be performed are displayed in different colors , and an imaging step of imaging an object to be operated, and a captured image of the display object and the object Then, it is determined whether the shape, color, or size of the object satisfies a condition indicated by the operation information, and when the condition is satisfied, an operation indicated by the operation information is performed on the object. And an operation execution step. In the operation execution step, the operation information is displayed by the operation information displayed in the first color. When the condition indicated by the operation information displayed in the first color is satisfied, whether the condition indicated by the operation information displayed in the second color is satisfied or not. If the condition indicated by the operation information displayed in the first color is not satisfied and the condition indicated by the operation information displayed in the second color is satisfied, the second A program for executing an operation indicated by operation information displayed in color is provided.

  ADVANTAGE OF THE INVENTION According to this invention, a mobile body can operate | move without remote-controlling a mobile body in real time by operation of an operator, and it becomes easy for a human to understand what operation | movement a mobile body performs.

The figure which illustrates the outline | summary of the control system 1 which concerns on one Embodiment of this invention. FIG. 3 is a diagram illustrating a hardware configuration of a moving body 20; The figure which illustrates the functional structure of the moving body 20. The figure which illustrates the display thing 10. The figure which illustrates the display thing 10. 5 is a flowchart illustrating an operation of the moving body 20. The figure which illustrates the relationship between the movement path | route of the moving body 20, and the display content of a display object.

DESCRIPTION OF SYMBOLS 1 ... Control system, 10 ... Display object 10 ... Moving body, 21 ... Imaging part, 22 ... Operation execution part, 221 ... Image analysis part, 222 ... Moving part, 223 ... Processing part, 201 ... CPU, 202 ... RAM, 203 .. ROM, 204 auxiliary storage device, 205 communication IF, 206 camera, 207 positioning device, 208 rotation mechanism, 209 processing mechanism.

1. Configuration FIG. 1 is a diagram illustrating an outline of a control system 1 according to an embodiment of the present invention. The control system 1 includes a plurality of display objects 10 and a moving body 20. The moving body 20 is a device that is movable and has an imaging function. In the present embodiment, the moving body 20 is a flyable rotary wing machine called a drone or a multicopter. The display object 10 is arranged at a position for notifying the moving body 20 of an operation performed by the moving body 20. On the display object 10, operation information indicating an operation performed by the moving body 20 is displayed by at least one of a character, a symbol, and a figure that can be interpreted by a human (that is, the meaning is understood only by a human). . The display form is, for example, that the motion information is written on the medium by hand, that the motion information is printed on the medium, that the motion information is handwritten or printed, that a sticker is attached to the structure, Alternatively, the information may be in any form, for example, the operation information is displayed on an optical display device.

  FIG. 2 is a diagram illustrating a hardware configuration of the moving object 20. The mobile unit 20 includes a CPU 201 (Central Processing Unit), a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an auxiliary storage device 204, a communication IF 205, a camera 206, a positioning device 207, a rotation mechanism 208, and a processing mechanism 209. Having. The CPU 201 is a processor that performs various operations. The RAM 203 is a volatile memory that functions as a work area when the CPU 201 executes a program. The ROM 202 is a non-volatile memory that stores, for example, a program and data used for activating the mobile unit 20. The auxiliary storage device 204 is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores programs and data used in the mobile unit 20. When the CPU 201 executes this program, the moving body 20 functions as a computer device, and the functions shown in FIG. 3 described later are realized. The communication IF 205 performs communication via a network (not shown) when the mobile unit 20 performs data communication with an external device. This communication may be wireless communication or wired communication. The camera 206 captures an image of a space vertically below the moving body 20, for example, and generates image data indicating the captured image. The positioning device 207 is, for example, a GPS (Global Positioning System) unit, and detects the position of the moving body 20. The rotation mechanism 208 is a unit that generates a lift and a propulsion in the moving body 20, and includes a propeller, a shaft, a motor, and other drive mechanisms. The processing mechanism 209 is a mechanism that performs a predetermined process, and includes, for example, a tank that stores the pesticide and a spraying device that sprays the pesticide if the operation performed by the moving body 20 is the spraying of the pesticide.

  FIG. 3 is a diagram illustrating a functional configuration of the moving body 20. The moving body 20 includes an imaging unit 21 and an operation executing unit 22 including an image analyzing unit 221, a moving unit 222, and a processing unit 223. The imaging unit 21 is realized by the camera 206, the image analysis unit 221 is realized by the CPU 201, the moving unit 222 is realized by the CPU 201, the positioning device 207, and the rotating mechanism 208. The processing unit 223 is the CPU 201 of the moving body 20, the camera 206, This is realized by the positioning device 207, the rotation mechanism 208, and the processing mechanism 209. The imaging unit 21 is a unit that performs imaging, and for example, images the display object 10 and generates captured image data indicating the captured image. The image analysis unit 221 analyzes captured image data using an image recognition technology such as an OCR (Optical Character Recognition / Reader) or a language processing technology, and specifies the contents of the operation information. The moving unit 222 and the processing unit 223 perform an operation (for example, moving or spraying a pesticide) according to the specified operation information.

  FIG. 4 is a diagram illustrating the display object 10. In the example of FIG. 4A, the altitude and speed when the moving body 20 moves and the processing (still image capturing every one second) performed while the moving body 20 moves are displayed in characters, and the moving direction is indicated by an arrow. It is indicated by a symbol. The moving body 20 reads the moving direction, altitude and speed, and the processing performed during the movement from the display object 10 and executes these operations.

  In the example of FIG. 4B, the direction in which the moving body 20 moves and the processing performed during the movement (moving while moving) are displayed in characters. In the example of FIG. 4C, the direction in which the moving body 20 moves is indicated by an arrow symbol, the processing (moving image shooting) performed during the movement is indicated by characters, and the position of the next display object 10 is further indicated. The distance up to (100 m) is displayed in characters. The moving body 20 reads the moving direction and the processing to be performed during the movement from such a display object 10, moves toward the next position of the display object 10 while performing these operations, and moves by a specified distance. Then, the display object 10 is searched.

  In the example of FIG. 4D, the processing (bag drop) performed by the moving body 20 at the position of the display object 10 is displayed in characters. The moving body 20 reads the processing from the display object 10 and executes this operation. In the example of FIG. 4E, the processing (pesticide spraying) performed by the moving body 20 and the positional range in which the operation is performed (within a circle having a radius of 100 m around the position of the display object 10) are displayed by characters. ing. The moving body 20 reads the processing and the position range from the display object 10 and executes the operation in the position range. In the example of FIG. 4F, the processing (pesticide spraying) performed by the moving body 20 is displayed in characters, and the positional range in which the operation is performed is displayed in characters, symbols, and graphics.

  In the example of FIG. 5A, the restriction (prohibition of entry) on the movement of the moving body 20 is displayed in characters. The moving body 20 reads the operation restriction from the display object 10 and changes the traveling direction up to that point. In the example of FIG. 5B, the restriction on the movement of the moving body 20 (prohibition of entry) is displayed by characters, and the positional range of the restriction is displayed by characters, symbols, and graphics. In the example of FIG. 5C, the restriction on the operation of the moving body 20 (not to spray the pesticide) and the positional range of the restriction are displayed by characters. The moving body 20 reads the operation restriction from the display object 10 and stops the application of the pesticide in the limited position range even if the application of the pesticide has been performed until then.

  In the example of FIG. 5D, the condition (the occurrence of the pest in the area A) and the operation (the spraying of the pesticide) performed by the moving body 20 when the condition is satisfied are displayed in characters. The position of the area A is stored in the moving body 20 in advance, and the moving body 20 reads a condition and an operation from the display object 10 and executes the operation when the condition is satisfied. More specifically, the condition corresponding to the analysis result of the captured image is stored in the auxiliary storage device 204 of the moving body 20. For example, if the shape (color, size, etc.) of the object (image object of the plant) included in the captured image is in a predetermined state, the symptom may be a pest occurrence symptom, a water shortage symptom, a fertilizer shortage symptom, a growth completion symptom, etc. Is stored. The operation execution unit 22 of the moving body 20 analyzes the shape, color, size, and the like of the object included in the captured image using an image recognition technique, and stores the analysis result in the condition (pest occurrence symptom) stored in the auxiliary storage device 204. , Water deficiency symptom, fertilizer deficiency symptom, growth completion symptom ...). Then, when the analysis result matches the condition specified by the operation information, the operation execution unit 22 executes the operation specified by the operation information.

  In the example of FIG. 5E, the condition (the occurrence of the pest in the area A) and the operation (the spraying of the pesticide) performed by the moving body 20 when the condition is satisfied are represented by a character of a certain color α. In addition, the condition (the occurrence of the pest in the area B) and the operation (the pesticide spraying) performed by the moving body 20 when the condition is satisfied are displayed in characters of different colors β. ing. Here, the operation information displayed in the character of the color α has higher priority than the operation information displayed in the character of the color β. Therefore, when the mobile unit 20 reads the condition and the operation from the display object 10, it checks the pest occurrence status in the area A and the area B, and if the pest is generated in the area A, the pest occurrence status in the area B is determined. Regardless, the operation (pesticide spraying) is first executed for the area A. On the other hand, when pests are not generated in area A and pests are generated in area B, or when pests are generated in area A and pesticides in area A still have surplus pesticides, , The moving body 20 performs an operation (agrochemical spraying) for the area B.

  In the example of FIG. 5F, the condition and the end of the operation performed by the moving body 20 when the condition is satisfied are both displayed in characters.

  As described above, the operation information may indicate an operation related to the movement of the moving body 20. In addition, the operation information may indicate an operation (for example, imaging, dropping of luggage, spraying of agrochemicals, etc.) performed by the moving body 20 on the surrounding environment. The operation information indicates a condition and an operation to be performed when the condition is satisfied. The operation execution unit 22 determines whether or not a condition indicated by the operation information is satisfied. The operation shown may be performed. Further, the operation information may indicate a restriction on the operation of the moving body 20. Further, the motion information may indicate a range of a position related to the motion of the moving body 20. Further, a plurality of pieces of operation information are displayed in different colors on the display object 10, and the operation execution unit 22 may execute an operation according to a color for displaying each operation information in the captured image.

2. Operation FIG. 6 is a flowchart illustrating an operation of the moving body 20. In FIG. 5, the moving unit 222 of the moving body 20 drives the rotating mechanism 208 to take off from a predetermined base facility and move the moving body 20 in accordance with an instruction given in advance (step S11). This instruction includes position information indicating the position of the display object 10 to be imaged first by the moving body 20, and the moving unit 222 moves the moving body 20 with the position as a destination.

  The moving unit 222 periodically detects the position of the moving body 20, and when the detected position matches the position indicated by the position information, the imaging unit 24 searches for the display object 10 while capturing an image. The display object 10 found as a result is imaged (step S12).

  The image analysis unit 25 analyzes the display object 10 included in the captured image (Step S13), and determines an operation indicated by the operation information displayed on the display object 10 (Step S14). The operation execution unit 22 executes the determined operation (Step S15). Thereafter, the processing of steps S12 to S15 is repeated until all the operations are completed (step S16; NO). When all the operations are completed (Step S16; YES), the processing of the moving body 20 ends.

  FIG. 7 is a diagram illustrating a relationship between a moving route of the moving body 20 and display contents of a display object. As shown in the figure, in the movement route of the moving body 20 from the start point S to the goal point G, a point that is a key point (for example, a point separated by a certain distance from the display object 10 captured last time, The display object 10 is arranged at a point where the operation of the moving object 20 is changed), and the moving object 20 displays the operation information (the arrow symbol indicating the moving direction in the example of FIG. 7) displayed on the display object 10. ) (Moves in the example of FIG. 7).

  According to the present embodiment described above, the mobile object 20 can analyze the display object and operate autonomously by merely arranging the display object 10 including the operation information for specifying the operation in each place. Becomes This motion information is displayed not by coded information (two-dimensional bar code or the like) that cannot be interpreted by humans, but by at least one of characters, symbols, and graphics that can be interpreted by humans. It becomes easy to instruct or instruct the mobile unit 20 to enter or input information by itself. In addition, if a person around the moving body 20 does not roughly understand how the moving body 20 operates, it is expected that a sudden operation by the moving body 20 may cause a dangerous situation. Is done. On the other hand, in the present embodiment, a person around the moving body 20 can easily understand what kind of operation the moving body 20 performs by looking at the display object 10. Is reduced.

3. Modifications The present invention is not limited to the above-described embodiment, and various modifications can be made. Hereinafter, some modified examples will be described. Two or more of the following modifications may be used in combination.

3-1. Modification 1
The operation execution unit 22 may recognize a handwriting in a captured image and execute an operation according to the handwriting. Specifically, a person who can give motion information to the moving object 20 is limited to one or more specific persons, and the characteristics of the handwriting of the person are stored in the auxiliary storage device 204. . The operation execution unit 22 recognizes a handwriting indicating the operation information in the captured image, and when the handwriting matches the one stored in the auxiliary storage device 204, executes the operation specified by the operation information. On the other hand, when the handwriting indicating the operation information in the captured image does not match the one stored in the auxiliary storage device 204, the operation execution unit 22 does not execute the operation specified by the operation information. By doing so, it is possible to eliminate unauthorized acts such as an unauthorized third person instructing the mobile unit 20 to perform an operation.

3-2. Modification 2
The display form of the operation information displayed on the display object 10 illustrated in FIG. 4 is merely an example, and any method may be used as long as the operation information is displayed by at least one of a character, a symbol, and a graphic that can be interpreted by a human. May be something. Note that the condition included in the operation information may specify a condition (for example, temperature, humidity, and precipitation) relating to the natural environment. In this case, the moving body 20 acquires and acquires the values (temperature, humidity, and precipitation) relating to the natural environment specified by these conditions from, for example, a sensor included in the moving body 20 itself or a sensor installed on the ground. When the value satisfies the above condition, the operation specified by the operation information is performed.

3-3. Modification 3
The moving object in the present invention is not limited to the moving object 20 flying in the air like the rotary wing aircraft exemplified in the embodiment, and may be, for example, a moving object moving on the ground or underwater.

3-4. Modification 4
A part of the functional configuration illustrated in FIG. 3 may be implemented in a device other than the mobile unit 20. For example, the image analysis unit 221 may be realized by an external device different from the mobile unit 20 (for example, a computer device capable of communicating with the mobile unit 20). In this case, the captured image data captured by the imaging unit 21 is transmitted to the external device via the communication IF 205 of the mobile unit 20. In the external device, the image analysis unit analyzes the display object 10 included in the captured image and displays the image. The operation indicated by the operation information displayed on the object 10 is determined, and the operation is notified to the mobile unit 20. In response to this notification, the operation execution unit 22 of the mobile unit 20 executes the notified operation. Thus, the present invention can be grasped as an invention of a control system. That is, the control system according to the present invention includes an imaging unit that captures an image of a display object in which operation information indicating an operation performed by the moving object is displayed by at least one of a character, a symbol, and a figure that can be interpreted by a human. And an operation execution unit that analyzes the captured image and causes the mobile object to execute the operation indicated by the operation information.

  The program executed by the CPU 201 may be provided by a storage medium such as an optical disk, a magnetic disk, or a semiconductor memory, or may be downloaded via a communication line such as the Internet. Further, these programs may not execute all the steps described in the embodiment.

Claims (5)

The operation information indicating the operation performed by the moving object is at least one of a character, a symbol, and a figure that can be interpreted by a human, and the condition and the plurality of operation information indicating the operation performed when the condition is satisfied are different colors. A display object displayed in , and an imaging unit for imaging an object to be subjected to the operation ,
Analyzing a captured image of the display object and the object , determining whether the shape, color, or size of the object satisfies a condition indicated by the operation information, and when the condition is satisfied, An operation execution unit that executes an operation indicated by the operation information on the object ,
When the condition indicated by the operation information displayed in the first color is satisfied, the operation execution unit determines whether the condition indicated by the operation information displayed in the second color is satisfied. Regardless, execute the operation indicated by the operation information displayed in the first color,
If the condition indicated by the operation information displayed in the first color is not satisfied and the condition indicated by the operation information displayed in the second color is satisfied, the display is performed in the second color A moving body that performs an operation indicated by the obtained operation information. The condition is indicated by the character,
The operation execution unit,
In the case where the condition indicated by the operation information is satisfied, and when the handwriting of a character recognized from the display object matches the handwriting stored in the captured image,
The mobile object according to claim 1, wherein the mobile object performs an operation indicated by the operation information . The operation information indicating the operation performed by the moving object is at least one of a character, a symbol, and a figure that can be interpreted by a human, and the condition and the plurality of operation information indicating the operation performed when the condition is satisfied are different colors. A display object displayed in , and an imaging unit for imaging an object to be subjected to the operation ,
Analyzing a captured image of the display object and the object , determining whether the shape, color, or size of the object satisfies a condition indicated by the operation information, and when the condition is satisfied, An operation execution unit that executes an operation indicated by the operation information on the object ,
When the condition indicated by the operation information displayed in the first color is satisfied, the operation execution unit determines whether the condition indicated by the operation information displayed in the second color is satisfied. Regardless, execute the operation indicated by the operation information displayed in the first color,
If the condition indicated by the operation information displayed in the first color is not satisfied and the condition indicated by the operation information displayed in the second color is satisfied, the display is performed in the second color A control system for executing an operation indicated by the obtained operation information. The operation information indicating the operation performed by the moving object is at least one of a character, a symbol, and a figure that can be interpreted by a human, and the condition and the plurality of operation information indicating the operation performed when the condition is satisfied are different colors. An imaging step of imaging a display object displayed in and an object to be subjected to the operation ,
Analyzing a captured image of the display object and the object , determining whether the shape, color, or size of the object satisfies a condition indicated by the operation information, and when the condition is satisfied, An operation execution step of executing an operation indicated by the operation information on the object ,
In the operation execution step, when the condition indicated by the operation information displayed in the first color is satisfied, it is determined whether the condition indicated by the operation information displayed in the second color is satisfied. Regardless, execute the operation indicated by the operation information displayed in the first color,
If the condition indicated by the operation information displayed in the first color is not satisfied and the condition indicated by the operation information displayed in the second color is satisfied, the display is performed in the second color A control method for performing an operation indicated by the obtained operation information. In the computer that the mobile has,
The operation information indicating the operation performed by the moving object is at least one of a character, a symbol, and a figure that can be interpreted by a human, and the condition and the plurality of operation information indicating the operation performed when the condition is satisfied are different colors. An imaging step of imaging a display object displayed in and an object to be subjected to the operation ,
Analyzing a captured image of the display object and the object , determining whether the shape, color, or size of the object satisfies a condition indicated by the operation information, and when the condition is satisfied, Executing an operation indicated by the operation information on the object , and
In the operation execution step, when the condition indicated by the operation information displayed in the first color is satisfied, it is determined whether the condition indicated by the operation information displayed in the second color is satisfied. Regardless, execute the operation indicated by the operation information displayed in the first color,
If the condition indicated by the operation information displayed in the first color is not satisfied and the condition indicated by the operation information displayed in the second color is satisfied, the display is performed in the second color A program for executing the operation indicated by the received operation information.

Images