JP7584124B2 - Marker, information processing device, and program - Google Patents

Description

The present invention relates to a marker, an information processing device, and a program.

Research and development is being conducted on information processing devices that read information indicated by markers included in an image captured by an imaging unit equipped in a moving object and control the moving object. The moving object may be, for example, a drone or a robot.

In this regard, QR Code (registered trademark) is known as a marker that indicates information that can be read by an information processing device (see Patent Document 1). Another such marker is the ArUco marker (see Non-Patent Document 1).

Japanese Patent Application Publication No. 07-254037

S. Garrido-Jurado, et al., "Generation of fiducial marker dictionaries using mixed integer linear programming," Pattern Recognition, Volume 51, March 2016, Pages 481-491.

Here, the more complex the operation of the moving object, the greater the amount of information that the information processing device reads from the markers tends to be. When the amount of information that the information processing device is to read is large, one marker may not be able to display all of the information. To solve this problem, there is a method of having the information processing device read information indicated by each of two or more markers. However, with this method, the number of markers increases as the amount of information that the information processing device is to read increases. This leads to an increase in the area occupied by the markers, which is undesirable. As a method for suppressing the increase in the number of markers, a method is also known in which two or more markers having the same encoding type are superimposed on each other with different hues to form a single marker. However, with this method, the detection rate of information indicated by any of the two or more markers superimposed as a single marker may decrease. As a result, the information processing device may not be able to operate the moving object with precision.

The present invention was made in consideration of these circumstances, and aims to provide a marker, an information processing device, and a program that can accurately detect more information from an image in which the markers are captured, while suppressing an increase in the number of markers.

The present invention includes the following aspects.
[1] A marker in which a first marker indicating first information and a second marker indicating second information are superimposed, the first marker reflecting electromagnetic waves in a predetermined first wavelength band and indicating the first information by a predetermined first type of encoding, and the second marker reflecting electromagnetic waves in a second wavelength band different from the first wavelength band and indicating the second information by a second type of encoding different from the first type.
[2] At least one of the electromagnetic waves in the first wavelength band and the electromagnetic waves in the second wavelength band is visible light, and a hue of the first marker is different from a hue of the second marker.
[3] One of the hue of the first marker and the hue of the second marker is a hue located at a first vertex of three vertices of an equilateral triangle inscribed in a specified hue circle, and the other of the hue of the first marker and the hue of the second marker is a hue located at a second vertex of the three vertices, which is different from the first vertex, in a specified hue circle.
[4] The first marker is an ArUco marker.
[5] The second marker is a QR code.
[6] In addition to the first marker and the second marker, one or more markers are superimposed.
[7] An information processing device comprising: a control unit that detects the first information and the second information based on an image in which the marker described above is captured.
[8] The control unit detects the first information indicated by the first marker based on the captured image and detects a contour of the first marker, and detects the second information indicated by the second marker based on the detected contour of the first marker and the captured image.
[9] An information processing device comprising: a control unit that detects the first information and the second information based on an image in which the marker described above is captured.
[10] The control unit detects the first information indicated by the first marker based on the captured image and detects a contour of the first marker, and detects the second information indicated by the second marker based on the detected contour of the first marker and the captured image.
[11] The information processing device further includes an imaging unit having a lens to which a filter that transmits at least one of electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, and the control unit detects the first information and the second information based on the captured image captured by the imaging unit.
[12] The information processing device further includes a communication unit that communicates with a moving body, the moving body including an imaging unit having a lens to which a filter that transmits electromagnetic waves in at least one of the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, the first information being information indicating a relative position and attitude between the imaging unit and the marker, and the second information being information indicating the position and attitude of the marker in a predetermined coordinate system, the control unit detecting the first information and the second information based on the captured image captured by the imaging unit, calculating the position and attitude of the moving body in the predetermined coordinate system based on the detected first information and second information, and controlling the moving body via the communication unit based on the calculated position and attitude of the moving body.
[13] The moving object is a drone or a robot.
[14] After detecting the first information and the second information, the control unit stores in a memory unit history information that corresponds information indicating the first marker from which the first information was detected and the detected second information.
[15] A program for causing a computer to execute a detection step of detecting the first information and the second information based on a captured image of the marker described above.

According to the present invention, it is possible to accurately detect more information from an image in which the markers are captured while suppressing an increase in the number of markers.

1 is a diagram illustrating an example of a configuration of a mobile object control system 1 according to an embodiment. FIG. 2 is a diagram showing an example of the configuration of a marker MK. 13A and 13B are diagrams showing other examples of the configuration of the marker MK. FIG. 13 is a diagram showing yet another example of the configuration of the marker MK. FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device 20. FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device 20. 1 is a diagram showing an example of a process flow in which an information processing device 20 controls a moving object 10. FIG. 1 is a diagram showing an example of the relationship between a trajectory along which a moving object 10 is moved and a position calculated by an information processing device 20. FIG. FIG. 9 is a diagram showing an example of the relationship in the height direction, among the relationships shown in FIG. 8 . 1 is a diagram showing an example of an information processing device 20 including an imaging unit 11 having a filter FT attached to a lens. 13 is a diagram showing an example of an image in which a marker MKA is captured by the imaging unit 11 when a filter FT is not attached to the imaging unit 11. FIG. 13 is a diagram showing an example of an image of a marker MKA captured by the imaging unit 11 when a filter FT that transmits only light in the red wavelength band is attached to the imaging unit 11. FIG. 13 is a diagram showing an example of an image of a marker MKA captured by the imaging unit 11 when a filter FT that transmits only the blue wavelength band is attached to the imaging unit 11. FIG.

<Embodiment>
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the embodiment, the marker is a figure, a mark, etc. that indicates various information by a predetermined type of encoding. The marker is, for example, an ArUco marker, a QR code, a barcode, a concentric circle marker, etc., but is not limited to these. Here, the ArUco marker, the QR code, the barcode, and the concentric circle marker are markers with different types of encoding. In the embodiment, a plurality of markers with different types of encoding indicate various information by different types of encoding. In the embodiment, a plurality of markers with the same type of encoding indicate various information by the same type of encoding. For example, the ArUco marker and the QR code are markers with different types of encoding. In addition, for example, two ArUco markers are markers with the same type of encoding.

<Configuration of the mobile control system>
Hereinafter, a configuration of a mobile object control system 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a mobile object control system 1 according to an embodiment.

The mobile object control system 1 includes a mobile object 10 and an information processing device 20. The mobile object 10 includes an imaging unit 11. The mobile object control system 1 may include other devices in addition to the mobile object 10 and the information processing device 20. In the mobile object control system 1, the mobile object 10 may be integrated with the information processing device 20. In the mobile object control system 1, the mobile object 10 may have some of the functions of the information processing device 20 described below.

The mobile object control system 1 controls the mobile object 10 using an information processing device 20. For example, the mobile object control system 1 moves the mobile object 10 along a predetermined trajectory under the control of the information processing device 20.

The moving body 10 may be any object that can move a part or the whole of the moving body 10 under the control of the information processing device 20. In the example shown in FIG. 1, the moving body 10 is a drone. Note that the moving body 10 may be a robot or the like instead of a drone.

The mobile body 10 is connected to the information processing device 20 wirelessly so as to be able to communicate with it. Note that the mobile body 10 may also be configured to be connected to the information processing device 20 wirelessly so as to be able to communicate with it.

The imaging unit 11 is a camera equipped with an imaging element that converts focused electromagnetic waves into an electrical signal. In the following, as an example, a case will be described in which the imaging unit 11 is a camera equipped with a CCD (Charge Coupled Device), CMOS (Complementary Metal Oxide Semiconductor), or the like as an imaging element that converts focused visible light into an electrical signal. The imaging unit 11 is wirelessly connected to the information processing device 20 so as to be able to communicate with it. Note that the imaging unit 11 may also be configured to be connected to the information processing device 20 so as to be able to communicate with it via a wire.

The imaging unit 11 is capable of capturing an image of a range corresponding to the position and attitude of the moving body 10. The imaging unit 11 captures an image of a range that the imaging unit 11 can capture under the control of the information processing device 20. The imaging unit 11 transmits the captured image to the information processing device 20.

The information processing device 20 is, for example, a multi-function mobile phone terminal (smartphone). Note that instead of a multi-function mobile phone terminal, the information processing device 20 may be another information processing device such as a tablet PC (Personal Computer), a notebook PC, a PDA (Personal Digital Assistant), a mobile phone terminal, a desktop PC, or a workstation.

The information processing device 20 causes the imaging unit 11 to capture an image of a range including the marker MK. The marker MK is, for example, a marker that indicates information indicating the relative position and orientation of the imaging unit 11 and the marker MK, and information indicating the position and orientation of the marker MK in a predetermined coordinate system. The predetermined coordinate system is, for example, a world coordinate system. Note that the predetermined coordinate system may also be another coordinate system that is stationary with respect to the marker MK.

The information processing device 20 detects information indicated by the marker MK, for example, based on an image including the marker MK among the images captured by the imaging unit 11. The information processing device 20 calculates the position and attitude of the moving body 10 in a predetermined coordinate system based on the detected information. The information processing device 20 moves the moving body 10 along the trajectory based on the calculated position and attitude of the moving body 10 and pre-stored trajectory information. The trajectory information is information indicating the above-mentioned predetermined trajectory. Note that the information processing device 20 may be configured to cause the moving body 10 to perform another operation based on the captured image including the marker MK. In this case, the information processing device 20 may be configured to calculate the position and attitude of the moving body 10 in the predetermined coordinate system, or may be configured not to calculate the position and attitude of the moving body 10 in the predetermined coordinate system.

Here, the more complex the operation that the moving body 10 is made to perform based on the information indicated by the marker included in the captured image, the greater the amount of information that the information processing device 20 reads from the marker. When the amount of information that the information processing device 20 is made to read is large, one marker may not be able to indicate all of the information. To solve this problem, there is a method in which the information processing device 20 reads information indicated by each of two or more markers. However, with this method, the number of markers increases as the amount of information that the information processing device 20 is made to read increases. As a method for suppressing the increase in the number of markers, a method is also known in which two or more markers having the same encoding type are superimposed with different hues to form a single marker. However, with this method, the detection rate of information indicated by any of the two or more markers superimposed as a single marker may decrease. In this case, the information processing device 20 may not be able to operate the moving body 10 with high accuracy.

Therefore, the marker MK is a marker on which markers indicating two or more pieces of information are superimposed. However, the two or more markers superimposed on the marker MK include at least markers indicating information by two different types of encoding. Therefore, for example, the two or more markers superimposed on the marker MK may be markers with different types of encoding. Also, for example, the two or more markers superimposed on the marker MK may include one or more markers indicating information by one of the two different types of encoding and one or more markers indicating information by the other of the two different types of encoding. Also, each of the two or more markers superimposed on the marker MK is an electromagnetic wave of a different wavelength band and is a marker that reflects an electromagnetic wave of a single predetermined wavelength band. Therefore, in the embodiment, it is not assumed that a color code is used as the two or more markers. Below, as an example, a case where the marker MK is a marker on which a first marker MK1 and a second marker MK2 are superimposed will be described. The first marker MK1 is a marker that reflects electromagnetic waves in a predetermined first wavelength band and indicates predetermined first information by a predetermined first type of encoding. The second marker MK2 is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates predetermined second information by a second type of encoding different from the first type. Thereby, the marker MK can indicate each of the first information and the second information as one marker without decreasing the detection rate of each of the two pieces of information, the first information and the second information. As a result, the marker MK can accurately detect more information from the captured image in which the marker MK is captured while suppressing an increase in the number of markers. In other words, the information processing device 20 can accurately detect more information from the captured image in which the marker MK is captured while suppressing an increase in the number of markers. In this example, the first information is information indicating the relative position and orientation of the imaging unit 11 and the marker MK. In addition, in this example, the second information is information indicating the position and orientation of the marker MK in a predetermined coordinate system. In this case, the markers MK can move the moving body 10 along a predetermined trajectory with high precision while suppressing an increase in the number of markers. In other words, in this case, the information processing device 20 can move the moving body 10 along a predetermined trajectory with high precision while suppressing an increase in the number of markers.

The first marker MK1 may be a marker that indicates, as the first information, information other than information indicating the relative position and orientation of the imaging unit 11 and the marker MK. The second marker MK2 may be a marker that indicates, as the second information, information other than information indicating the position and orientation of the marker MK in a predetermined coordinate system. The marker MK may be configured to have one or more other markers different from each of the first marker MK1 and the second marker MK2 superimposed thereon, together with the first marker MK1 and the second marker MK2. Some or all of the one or more other markers may be markers that indicate various information by a type of encoding different from the type of encoding of each of the first marker MK1 and the second marker MK2, or may be markers that indicate various information by a type of encoding that is the same as the type of encoding of each of the first marker MK1 and the second marker MK2. Some or all of the one or more markers may be markers that have different types of encoding from each other, or may be markers that have the same type of encoding from each other.

The first marker MK1 has a first marker portion that reflects electromagnetic waves in a predetermined first wavelength band. The first marker MK1 may have a first background portion that is the background of the first marker MK1 in addition to the first marker portion, or may not have a first background portion. The first background portion may be configured to reflect electromagnetic waves in a wavelength band different from the second wavelength band, or may be a portion that hardly reflects electromagnetic waves. The first marker MK1 may have a first additional portion that is different from each of the first marker portion and the first background portion in addition to the first marker portion, or may not have a first additional portion. However, this portion reflects electromagnetic waves in a wavelength band different from each of the first wavelength band and the second wavelength band. The second marker MK2 has a second marker portion that reflects electromagnetic waves in a second wavelength band different from the first wavelength band. The second marker MK2 may have a second background portion that serves as the background of the second marker MK2 in addition to the second marker portion, or may not have a second background portion. The second background portion may be configured to reflect electromagnetic waves in a wavelength band different from the first wavelength band, or may be a portion that hardly reflects electromagnetic waves. Furthermore, the second marker MK2 may have a second additional portion that is additional to the second marker portion and the second background portion, in addition to the second marker portion, or may not have a second additional portion. However, the second additional portion reflects electromagnetic waves in a wavelength band different from the first wavelength band and the second wavelength band.

In the following, as an example, a case where the first wavelength band and the second wavelength band are wavelength bands of visible light will be described. In this case, the electromagnetic waves of the first wavelength band and the electromagnetic waves of the second wavelength band are visible light. The fact that the first wavelength band and the second wavelength band are different wavelength bands and that the first wavelength band and the second wavelength band are wavelength bands of visible light means that the hue of the first marker is different from the hue of the second marker. Here, the hue of the first marker refers to the hue of the first marker portion. The hue of the second marker refers to the hue of the second marker portion. At least one of the first wavelength band and the second wavelength band may be a wavelength band different from the wavelength band of visible light. For example, at least one of the first wavelength band and the second wavelength band may be an infrared wavelength band. In addition, the second wavelength band may be configured to overlap a part of the first wavelength band, or may not overlap the entire first wavelength band.

The following provides a detailed explanation of the configuration of the marker MK, the configuration of the information processing device 20, and the process by which the information processing device 20 controls the moving body 10.

<Marker MK configuration>

The configuration of the marker MK will be described below. FIG. 2 is a diagram showing an example of the configuration of the marker MK. In FIG. 2, the marker MKA is shown as an example of the marker MK. The marker MKA is a marker in which a first marker MK1A, which is an example of the first marker MK1, and a second marker MK2A, which is an example of the second marker MK2, are superimposed. Note that in FIG. 2, differences in hue are indicated by differences in hatching.

The first marker MK1A is an ArUco marker. That is, the first marker MK1A is an example of a first marker MK1 that has a first marker portion and does not have a first background portion or a first additional portion. The first marker MK1A is a marker with a red hue. That is, the hue of the first marker portion of the first marker MK1A is red. Note that instead of an ArUco marker, the first marker MK1A may be any marker that indicates the first information by a type of encoding different from the second type. Also, the first marker MK1A does not necessarily have no first background portion, and the first background portion may be interpreted as being transparent, and may be interpreted as having been removed when superimposed on the second marker MK2A.

The second marker MK2A is a QR code. That is, the second marker MK2A is an example of a second marker MK2 that has a second marker portion and a second background portion, and does not have a second additional portion. The second marker MK2A is a marker with a green hue. That is, the hue of the second marker portion of the second marker MK2A is green. In the example shown in FIG. 2, the hue of the second background portion is black, which hardly reflects electromagnetic waves (in FIG. 2, the second background portion is not hatched to prevent the figure from becoming too complicated). The second marker MK2A may be any marker that indicates the second information by a type of encoding different from the first type, instead of a QR code.

As shown in FIG. 2, in the marker MKA, the hue of the first marker MK1A is different from the hue of the second marker MK2A. That is, in the marker MKA, the hue of the first marker portion of the first marker MK1A is different from the hue of the second marker portion and the second background portion of the second marker MK2A. Therefore, the information processing device 20 can detect the first information indicated by the first marker MK1A and the second information indicated by the second marker MK2A based on the captured image in which the marker MKA is captured. Here, in the marker MKA, the wider the width of the thinnest line among the lines constituting the first marker portion, the higher the detection rate of the first information from the first marker MK1A by the information processing device 20. Also, in the marker MKA, the wider the width of the thinnest line among the lines constituting the second marker portion, the higher the detection rate of the second information from the second marker MK2A by the information processing device 20. The width L1A indicated by the arrow shown in FIG. 2 is an example of the width of the thinnest line among the lines constituting the first marker portion. Furthermore, the width L2A indicated by the arrow in FIG. 2 is an example of the width of the thinnest line among the lines that make up the second marker portion. In the example shown in FIG. 2, the width L1A is thicker than the width L2A. That is, in this example, the detection rate of the first information from the first marker MK1A by the information processing device 20 is higher than the detection rate of the second information from the second marker MK2A by the information processing device 20.

Figure 3 is a diagram showing another example of the configuration of the marker MK. In Figure 3, the marker MKB is shown as an example of the marker MK. The marker MKB is a marker in which a first marker MK1B, which is an example of the first marker MK1, and a second marker MK2B, which is an example of the second marker MK2, are superimposed. Note that in Figure 3, differences in hue are indicated by differences in hatching.

The first marker MK1B is a concentric marker. That is, the first marker MK1B is an example of a first marker MK1 that has a first marker portion and does not have a first background portion or a first additional portion. The first marker MK1B is a marker with a red hue. That is, the hue of the first marker portion of the first marker MK1B is red. Note that instead of a concentric marker, the first marker MK1B may be any marker that indicates the first information by a type of encoding different from the second type. Also, the first marker MK1B does not necessarily have no first background portion, and the first background portion may be interpreted as being transparent, and may be interpreted as having been removed when superimposed on the second marker MK2B.

The second marker MK2B is a barcode. That is, the second marker MK2B is an example of a second marker MK2 that has a second marker portion and a second background portion, but does not have a second additional portion. The second marker MK2B is a marker with a blue hue. That is, the hue of the second marker portion of the second marker MK2B is blue. In the example shown in FIG. 3, the hue of the second background portion is purple (in FIG. 3, the second background portion is not hatched to prevent the diagram from becoming too cluttered). The second marker MK2B may be any marker that indicates the second information by a type of encoding different from the first type, instead of a barcode.

As shown in FIG. 3, in the marker MKB, the hue of the first marker MK1B is different from the hue of the second marker MK2B. That is, in the marker MKB, the hue of the first marker portion of the first marker MK1B is different from the hue of the second marker portion and the second background portion of the second marker MK2B. Therefore, the information processing device 20 can detect the first information indicated by the first marker MK1B and the second information indicated by the second marker MK2B based on the captured image in which the marker MKB is captured. Here, the width L1B indicated by the arrow shown in FIG. 3 is an example of the width of the thinnest line among the lines constituting the first marker portion. Also, the width L2B indicated by the arrow shown in FIG. 3 is an example of the width of the thinnest line among the lines constituting the second marker portion. In the example shown in FIG. 3, the width L1B is thicker than the width L2B. That is, in this example, the detection rate of the first information from the first marker MK1B by the information processing device 20 is higher than the detection rate of the second information from the second marker MK2B by the information processing device 20.

Figure 4 is a diagram showing yet another example of the configuration of the marker MK. In Figure 4, a marker MKC is shown as an example of the marker MK. The marker MKC is a marker in which a first marker MK1C, which is an example of the first marker MK1, a second marker MK2C, which is an example of the second marker MK2, and a third marker MK3C, which is a third marker, are superimposed. Note that, for convenience of explanation, the type of encoding of the third marker MK3C will be referred to as the third type below. Also, for convenience of explanation, the information indicated by the third marker MK3C will be referred to as the third information below. Also, in Figure 4, differences in hue are indicated by differences in hatching.

The first marker MK1C is an ArUco marker. That is, the first marker MK1C is an example of a first marker MK1 that has a first marker portion and does not have a first background portion or a first additional portion. The first marker MK1C is a marker with a red hue. That is, the hue of the first marker portion of the first marker MK1C is red. Note that instead of an ArUco marker, the first marker MK1C may be any marker that indicates the first information by a type of encoding different from each of the second and third types. Also, the first marker MK1C does not necessarily have no first background portion, and the first background portion may be interpreted as being transparent, or the first background portion may be interpreted as being removed when superimposed on the second marker MK2C.

The second marker MK2C is a QR code. That is, the second marker MK2C is an example of a second marker MK2 that has a second marker portion and a second background portion, and does not have a second additional portion. The second marker MK2C is a marker with a blue hue. That is, the hue of the second marker portion of the second marker MK2C is blue. In the example shown in FIG. 4, the hue of the second background portion is black, which hardly reflects electromagnetic waves (in FIG. 4, the second background portion is not hatched to prevent the figure from becoming complicated). The second marker MK2C may be any marker that indicates the second information by a type of encoding different from each of the first and third types, instead of a QR code.

The third marker MK3C is a character code. The third marker MK3C has a character portion and no background portion. The third marker MK3C is a marker with a green hue. That is, the hue of the character portion of the third marker MK3C is green. The third marker MK3C may be any marker that indicates the third information by a type of encoding different from each of the first and second types instead of a character code. The third marker MK3C does not have no background portion, and the portion may be interpreted as being transparent, or the portion may be interpreted as being removed when superimposed on the first marker MK1C and the second marker MK2C.

As shown in FIG. 4, in the marker MKC, the hue of the first marker MK1C is different from the hue of the second marker MK2C, and the hue of the third marker MK3C. That is, in the marker MKC, the hue of the first marker portion of the first marker MK1B is different from the hue of each of the second marker portion and the second background portion of the second marker MK2B, and the hue of the character portion of the third marker MK3C. Therefore, the information processing device 20 can detect the first information indicated by the first marker MK1C, the second information indicated by the second marker MK2C, and the third information indicated by the third marker MK3C based on the captured image in which the marker MKC is captured. Here, the width L1B indicated by the arrow shown in FIG. 4 is an example of the width of the thinnest line among the lines constituting the first marker portion. Also, the width L2C indicated by the arrow shown in FIG. 4 is an example of the width of the thinnest line among the lines constituting the second marker portion. Furthermore, width L3C indicated by the arrow in FIG. 4 is an example of the width of the thinnest line among the lines constituting the character portion of the third marker MK3C. In the example shown in FIG. 4, width L1C is wider than width L2C and width L3C. Furthermore, width L2C is wider than width L3C. That is, in this example, the detection rate of the first information from the first marker MK1C by the information processing device 20 is higher than the detection rate of the second information from the second marker MK2C by the information processing device 20. Furthermore, in this example, the detection rate of the second information from the second marker MK2C by the information processing device 20 is higher than the detection rate of the third information from the third marker MK3C by the information processing device 20.

In this way, the marker MK is a marker in which two or more markers with different types of encoding are superimposed. This allows the marker MK to indicate two or more pieces of information. As a result, the information processing device 20 can detect two or more pieces of information from one marker MK. This leads to a reduction in the area and volume occupied by the marker. As a result, the marker MK can indicate two or more pieces of information and can increase the degree of freedom in installation. In addition, when markers indicating various information by existing types of encoding are used as the first marker MK1 and the second marker MK2 in the marker MK, existing algorithms and libraries can be used as algorithms and libraries for detecting the marker MK. In addition, for example, when a third marker such as the marker MKC is superimposed on the first marker MK1 and the second marker MK2, information indicating the detection method of the first marker MK1 can be made to be displayed on the third marker. In this case, instead of a marker indicating various information by existing types of encoding, a marker indicating various information by a newly developed type of encoding can be easily used as the first marker MK1. Furthermore, the information processing device 20 that detects the first information and the second information from the marker MK can improve the success rate of detection of the information indicated by the marker that is detected later, either the first marker MK1 or the second marker MK2, based on the outline of the marker that is detected earlier, either the first marker MK1 or the second marker MK2, as described below. Note that the number of markers superimposed as the marker MK may be four or more.

<Hardware configuration of information processing device 20>
The hardware configuration of the information processing device 20 will be described below with reference to Fig. 5. Fig. 5 is a diagram showing an example of the hardware configuration of the information processing device 20.

The information processing device 20 includes, for example, a CPU (Central Processing Unit) 21, a memory unit 22, an input reception unit 23, a communication unit 24, and a display unit 25. These components are connected to each other via a bus so that they can communicate with each other. Furthermore, the information processing device 20 communicates with each of the moving object 10 and the imaging unit 11 via the communication unit 24.

The CPU 21 is, for example, a processor that controls the entire information processing device 20. Note that the CPU 21 may be another processor such as an FPGA (Field-Programmable Gate Array). The CPU 21 executes various programs stored in the memory unit 22.

The storage unit 22 includes, for example, a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a random access memory (RAM), etc. Note that instead of being built into the information processing device 20, the storage unit 22 may be an external storage device connected via a digital input/output port such as a universal serial bus (USB). The storage unit 22 stores various information, various images, various programs, etc. that are processed by the information processing device 20.

The input reception unit 23 is an input device such as a keyboard, a mouse, or a touchpad. The input reception unit 23 may be a touch panel that is integrated with the display unit 25.

The communication unit 24 includes, for example, an antenna, a digital input/output port such as a USB, an Ethernet (registered trademark) port, etc.

The display unit 25 is a display device including, for example, a liquid crystal display panel or an organic EL (Electro Luminescence) display panel.

<Functional configuration of information processing device 20>
The functional configuration of the information processing device 20 will be described below with reference to Fig. 6. Fig. 6 is a diagram showing an example of the functional configuration of the information processing device 20.

The information processing device 20 includes a memory unit 22, an input receiving unit 23, a communication unit 24, a display unit 25, and a control unit 26.

The control unit 26 controls the entire information processing device 20. The control unit 26 includes, for example, an imaging control unit 261, an image processing unit 262, a position and orientation calculation unit 263, and a mobile body control unit 264. These functional units included in the control unit 26 are realized, for example, by the CPU 21 executing various programs stored in the storage unit 22. In addition, some or all of the functional units may be hardware functional units such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

The imaging control unit 261 causes the imaging unit 11 to capture an image of the range that the imaging unit 11 can capture.

The image processing unit 262 acquires the captured image captured by the imaging unit 11 from the imaging unit 11. The image processing unit 262 performs various image processing based on the acquired captured image. For example, the image processing unit 262 detects each of the first marker MK1 and the second marker MK2 included in the captured image based on the acquired captured image. Also, for example, the image processing unit 262 detects first information indicated by the detected first marker MK1. Also, for example, the image processing unit 262 detects second information indicated by the detected second marker MK2.

The position and orientation calculation unit 263 calculates the position and orientation of the moving body 10 in a specified coordinate system based on the results of image processing by the image processing unit 262.

The mobile body control unit 264 controls the mobile body 10 based on the position and orientation of the mobile body 10 calculated by the position and orientation calculation unit 263.

<Processing by the information processing device 20 to control the moving object 10>
Hereinafter, a process in which the information processing device 20 controls the moving body 10 will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a flow of a process in which the information processing device 20 controls the moving body 10. In the following, as an example, a case where the above-mentioned trajectory information is stored in the storage unit 22 at a timing before the processing of step S110 shown in FIG. 7 is performed will be described. In addition, in the following, as an example, a case where the position and attitude of the moving body 10 in a predetermined coordinate system at that timing coincide with a predetermined start position and a predetermined start attitude will be described. The predetermined start position is a position where the position of the moving body 10 coincides at a timing when the control of the moving body 10 by the information processing device 20 starts. The predetermined start attitude is a attitude where the attitude of the moving body 10 coincides at a timing when the control of the moving body 10 by the information processing device 20 starts. The position of the moving body 10 is represented, for example, by the position of the origin of a local coordinate system associated with the moving body 10 so as to move together with the moving body 10, among positions in a world coordinate system. The attitude of the moving body 10 is represented, for example, by the direction in which each coordinate axis of the local coordinate system faces among the directions in the world coordinate system. The position of the moving body 10 may be represented by another position according to the moving body 10. The attitude of the moving body 10 may be represented by another direction according to the moving body 10. In the following, as an example, a case where the marker MKA shown in FIG. 2 is used as the marker MK will be described. In the following, as an example, a case where the position and attitude of the moving body 10 match a predetermined start position and a predetermined start attitude, and the marker MKA is included in a range where the imaging unit 11 can capture images will be described.

The position and orientation calculation unit 263 reads out from the storage unit 22 the trajectory information that has been previously stored in the storage unit 22 (step S110).

Next, the imaging control unit 261 causes the imaging unit 11 to capture an image of the range that the imaging unit 11 can capture (step S120). As a result, the imaging unit 11 captures an image of the range including the marker MKA. Then, the image processing unit 262 receives the captured image captured by the imaging unit 11 from the imaging unit 11.

Next, the image processing unit 262 performs image separation based on the captured image received from the imaging unit 11 in step S120 (step S130). Here, image separation refers to image processing that separates the hue components of the first marker MK1A and the hue components of the second marker MK2A from the captured image to generate a first image including the first marker MK1A and a second image including the second marker MK2A.

Next, the image processing unit 262 detects the first information indicated by the first marker MK1A included in the first image based on the first image generated by image separation in step S130 (step S140). In this example, the first information is information indicating the relative position and orientation between the imaging unit 11 and the marker MKA, as described above. In this example, the imaging unit 11 is provided on the moving body 10. Therefore, the position and orientation of the imaging unit 11 can be approximately treated as the position and orientation of the moving body 10. That is, in step S140, the image processing unit 262 detects information indicating the relative position and orientation between the moving body 10 and the marker MKA as the first information. Note that the method of detecting the first information indicated by the first marker MK1A may be a known method or a method to be developed in the future.

Next, the image processing unit 262 detects the contour of the first marker MK1A included in the first image based on the first image (step S150). Note that the method for detecting the contour of the first marker MK1A may be a known method or a method to be developed in the future.

Next, the image processing unit 262 performs projective transformation of the image of the second marker MK2A included in the second image generated in step S130 based on the contour detected in step S150 (step S160). Here, the processing of step S160 will be described. As described above, the second marker MK2A has a lower detection rate than the first marker MK1A. For this reason, if the marker MKA is captured in the captured image from an oblique angle, the second marker MK2A may not be detectable. Therefore, the image processing unit 262 performs projective transformation of the image of the second marker MK2A included in the second image into an image of the second marker MK2A viewed from the front based on the contour. Note that the method of performing such projective transformation may be a known method or a method to be developed in the future.

Next, the image processing unit 262 detects the second information indicated by the second marker MK2A based on the second image after projective transformation in step S160 (step S170). In this example, the second information is information indicating the position and orientation of the marker MKA in a predetermined coordinate system, as described above. That is, in step S170, the image processing unit 262 detects information indicating the position and orientation of the marker MKA in a predetermined coordinate system as the second information. Note that the method for detecting the second information may be a known method or a method to be developed in the future.

Next, the position and orientation calculation unit 263 calculates the position and orientation of the moving body 10 in a predetermined coordinate system based on the first information detected in step S140 and the second information detected in step S170 (step S180).

Next, the mobile body control unit 264 moves the mobile body 10 along the trajectory indicated by the trajectory information based on the position and attitude of the mobile body 10 calculated in step S180 and the trajectory information read from the memory unit 22 in step S110 (step S190), and ends the process.

As described above, the information processing device 20 detects the first information and the second information from the marker MKA, and controls the moving body 10 based on the detected first information and second information. This allows the information processing device 20 to accurately detect more information from the captured image in which the marker MKA is captured, while suppressing an increase in the number of markers.

Here, FIG. 8 is a diagram showing an example of the relationship between the trajectory along which the moving body 10 has been moved and the position calculated by the information processing device 20. The horizontal axis of the graph shown in FIG. 8 indicates the position in the X-axis direction in a predetermined coordinate system. The vertical axis of the graph shown in FIG. 8 indicates the position in the Y-axis direction in a predetermined coordinate system. The solid line in the graph indicates the trajectory along which the moving body 10 has moved. The plots in the graph are positions calculated by the information processing device 20 based on the processing of the flowchart shown in FIG. 7, and indicate the position of the moving body 10 at each time while the moving body 10 is moving along the trajectory. As shown in FIG. 8, the information processing device 20 can accurately calculate the position of the moving body 10 on the XY plane of the predetermined coordinate axes based on the captured image in which the marker MK is captured.

Figure 9 is a diagram showing an example of the relationship in the height direction from the relationships shown in Figure 8. The horizontal axis of the graph shown in Figure 9 indicates the horizontal distance from the origin of a specified coordinate axis. The vertical axis of the graph indicates the position in the Z-axis direction in a specified coordinate system. The solid line in the graph indicates the trajectory of the moving body 10. The plots in the graph are positions calculated by the information processing device 20 based on the processing of the flowchart shown in Figure 7, and indicate the position of the moving body 10 at each time while the moving body 10 is moving along the trajectory. As shown in Figure 9, the information processing device 20 can accurately calculate the position of the moving body 10 in the Z-axis direction of the specified coordinate axis based on the captured image in which the marker MK is captured.

<First Modification of the Embodiment>
The information processing device 20 described above may be configured integrally with the imaging unit 11. In this case, the information processing device 20 may be configured to control the moving body 10, or may not be configured to control the moving body 10. In this case, the imaging unit 11 has, for example, a lens to which a filter FT (e.g., a bandpass filter or the like) that transmits electromagnetic waves of at least one of a first wavelength band and a second wavelength band through the lens can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal. Here, FIG. 10 is a diagram showing an example of an information processing device 20 including an imaging unit 11 with a filter FT attached to the lens.

When such a filter FT is attached to the imaging unit 11, the information processing device 20 can capture each of the first and second images described above without performing the image separation described above.

For example, FIG. 11 is a diagram showing an example of an image captured by the imaging unit 11 when the filter FT is not attached to the imaging unit 11, in which the marker MKA is captured. The image P1 shown in FIG. 11 is an example of the captured image. As shown in FIG. 11, in this case, the first marker MK1A and the second marker MK2A superimposed as the marker MKA can each be seen in FIG. 11. That is, in this case, the information processing device 20 must perform image separation when detecting the first information and the second information.

On the other hand, FIG. 12 is a diagram showing an example of an image in which the marker MKA is captured by the imaging unit 11 when a filter FT that transmits only the red wavelength band is attached to the imaging unit 11. Image P2 shown in FIG. 12 is an example of the captured image. In this case, in FIG. 12, the second marker MK2A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA can be seen, but the first marker MK1A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA cannot be seen. That is, in this case, the information processing device 20 can detect the second information based on the captured image without performing image separation.

Also, FIG. 13 is a diagram showing an example of an image in which the marker MKA is captured by the imaging unit 11 when a filter FT that transmits only the blue wavelength band is attached to the imaging unit 11. Image P3 shown in FIG. 13 is an example of the captured image. In this case, in FIG. 13, the first marker MK1A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA can be seen, and the second marker MK2A of the first marker MK1A and the second marker MK2A superimposed as the marker MKA cannot be seen. That is, in this case, the information processing device 20 can detect the first information based on the captured image without performing image separation.

Here, if the wavelength band of light transmitted by the filter FT can be changed, the information processing device 20 can detect the first information and the second information from the marker MKA without performing image separation, as described above. This means that the information processing device 20 can detect two or more pieces of information from the marker MK without installing a new application program, because existing application programs capable of detecting each of the first marker MK1A and the second marker MK2A are installed. Therefore, when obtaining information using the marker MK, the user of the information processing device 20 can select between a method of attaching the filter FT to the information processing device 20 and performing detection using an existing application program, and a method of performing detection using a new application program capable of performing image separation. This leads to improved convenience for the user, and is preferable.

<Modification 2 of the embodiment>
The filter FT described above may be configured to be attached to the imaging unit 11 provided in the moving body 10. In this case, the information processing device 20 can omit the process of step S130 in the flowchart shown in FIG. 7. However, in this case, in step S120, the information processing device 20 causes the imaging unit 11 to capture a range that the imaging unit 11 can capture, and receives each of the first image and the second image from the imaging unit 11, every time the wavelength band of light transmitted by the filter FT is changed. As a result, even if the information processing device 20 is configured not to perform image separation, it is possible to accurately detect more information from the captured image in which the marker MK is captured, while suppressing an increase in the number of markers.

<Modification 3 of the embodiment>
The information processing device 20 described above may be configured to detect the first information and the second information, and then store in the storage unit 22 history information that associates the information indicating the first marker MK1 from which the first information was detected with the detected second information. As a result, the information processing device 20 can read out the second information associated with the information indicating the first marker MK1 from which the first information was detected from the storage unit 22 after detecting the first information at the second detection of the marker MK. That is, in this case, the information processing device 20 does not need to perform processing to detect the second information. As a result, the information processing device 20 can reduce the load of the processing of the flowchart shown in FIG. 7 and can speed up the processing.

<Selection of the hue of the first marker and the hue of the second marker and detection accuracy>
In the above-described information processing device 20, when the first wavelength band and the second wavelength band are both visible light wavelength bands, the greater the difference between the hue of the first marker MK1 and the hue of the second marker MK2, the more accurately the information processing device 20 can detect the first information and the second information. Therefore, by selecting these two hues as described below, it is possible to prevent the detection accuracy of the first information and the second information by the information processing device 20 from decreasing.

First, one of the hues of the first marker MK1 and the second marker MK2 is assigned to an arbitrary hue in a predetermined hue circle. In the following, for convenience of explanation, this assigned hue will be referred to as the first hue. Then, an equilateral triangle inscribed in the hue circle is considered. This triangle is inscribed in the hue circle so that one of the vertices of the triangle is located in the first hue. Then, the hue in which one of the two vertices of the triangle other than the vertex located in the first hue is located is assigned as the other of the hue of the first marker MK1 and the hue of the second marker MK2. In this way, by selecting the hue of the first marker MK1 and the hue of the second marker MK2, the information processing device 20 can suppress a decrease in the detection accuracy of the first information and the second information by the information processing device 20. Note that the hue of the first marker MK1 and the hue of the second marker MK2 may be selected by another method.

In addition, the information processing device 20 described above may be configured to control a device that does not have a moving part under control of the information processing device 20, instead of the moving body 10. In this case, the device may be configured, for example, to have an imaging unit 11 attached to the device so that the imageable range can be changed, or to have an imaging unit 11 attached to the device so that the imageable range cannot be changed.

The marker MK described above may be attached to the moving body 10. In this case, the imaging unit 11 is installed at a position where it can capture an image of a range including the range in which the moving body 10 moves. One of the first marker MK1 and the second marker MK2 indicates the first marker MK1 indicating the individual information of the moving body 10. The other of the first marker MK1 and the second marker MK2 indicates information indicating the relative position and attitude of the imaging unit 11 and the marker MK as information indicating the relative position and attitude of the imaging unit 11 and the moving body 10. The information processing device 20 controls the moving body 10 based on these two pieces of information detected from the marker MK. Such a control method is often applied, for example, when the information processing device 20 controls (manages) individual moving bodies included in a group of multiple moving bodies. Furthermore, the process flow for detecting the first information and the second information in such a control method may be, for example, the process flow of steps S120 to S170 shown in FIG. 7, or may be another process flow capable of detecting the first information and the second information.

As described above, the markers according to the embodiment (in the examples described above, marker MK, marker MKA, marker MKB, and marker MKC) are markers in which a first marker indicating first information (in the examples described above, first marker MK1, first marker MK1A, first marker MK1B, and first marker MK1C) and a second marker indicating second information (in the examples described above, second marker MK2, second marker MK2A, second marker MK2B, and second marker MK2C) are superimposed, and the first marker is a marker that reflects electromagnetic waves in a predetermined first wavelength band and indicates the first information by a predetermined first type of encoding (in the examples described above, for example, an ArUco marker), and the second marker is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates the second information by a second type of encoding different from the first type (in the examples described above, for example, a QR code). This allows the markers to accurately detect more information from the captured image while preventing an increase in the number of markers.

The marker may also be configured such that at least one of the electromagnetic waves in the first wavelength band and the electromagnetic waves in the second wavelength band is visible light, and the hue of the first marker is a different hue from the hue of the second marker.

The marker may be configured such that one of the hue of the first marker and the hue of the second marker is a hue located at a first vertex of the three vertices of an equilateral triangle inscribed in a specified hue circle, and the other of the hue of the first marker and the hue of the second marker is a hue located at a second vertex, different from the first vertex, of the three vertices of the specified hue circle.

The marker may also be configured such that the first marker is an ArUco marker.

The marker may also be configured such that the second marker is a QR code.

The marker may also be configured such that, in addition to the first and second markers, one or more markers (in the example described above, the third marker MK3C) are superimposed.

The information processing device according to the embodiment (information processing device 20 in the example described above) also includes a control unit (control unit 26 in the example described above) that detects the first information and the second information based on an image in which the markers described above are captured. This allows the information processing device to cause the moving object to perform highly accurate operations while suppressing an increase in the number of markers.

In addition, the information processing device may be configured such that the control unit detects first information indicated by a first marker based on the captured image, detects the contour of the first marker, and detects second information indicated by a second marker based on the detected contour of the first marker and the captured image.

The information processing device may further include an imaging unit (in the example described above, imaging unit 11) having a lens to which a filter (in the example described above, filter FT) that transmits electromagnetic waves in at least one of the first and second wavelength bands can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, and the control unit may be configured to detect the first information and the second information based on an image captured by the imaging unit.

The information processing device may further include a communication unit (communication unit 24 in the example described above) that communicates with a moving body (moving body 10 in the example described above), the moving body may include an imaging unit having a lens to which a filter that transmits at least one of electromagnetic waves in a first wavelength band and a second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electrical signal, the first information is information indicating the relative position and attitude of the imaging unit and the marker, the second information is information indicating the position and attitude of the marker in a predetermined coordinate system (world coordinate system in the example described above), and the control unit may detect the first information and the second information based on the captured image captured by the imaging unit, calculate the position and attitude of the moving body in the predetermined coordinate system based on the detected first information and second information, and control the moving body via the communication unit based on the calculated position and attitude of the moving body.

The information processing device may also be configured so that the moving body is a drone or a robot.

In addition, the information processing device may be configured such that, after detecting the first information and the second information, the control unit stores in a storage unit (storage unit 22 in the example described above) history information that associates information indicating the first marker from which the first information was detected with the detected second information.

The embodiment of the present invention has been described above in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and may be changed, replaced, deleted, etc., without departing from the spirit of the present invention.

A program for implementing the functions of any of the components in the above-described device (e.g., information processing device 20, etc.) may be recorded on a computer-readable recording medium, and the program may be read and executed by a computer system. Note that the term "computer system" here includes hardware such as an OS (Operating System) and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CDs (Compact Disks)-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" refers to storage devices that hold a program for a certain period of time, such as volatile memory (RAM (Random Access Memory)) inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

The above program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The above program may be for implementing some of the above functions. Furthermore, the above program may be a so-called differential file (differential program) that can implement the above functions in combination with a program already recorded in the computer system.

1...mobile body control system, 10...mobile body, 11...imaging unit, 20...information processing device, 22...storage unit, 23...input reception unit, 24...communication unit, 25...display unit, 26...control unit, 261...imaging control unit, 262...image processing unit, 263...position and orientation calculation unit, 264...mobile body control unit, FT...filter, MK, MKA, MKB, MKC...marker, MK1, MK1A, MK1B, MK1C...first marker, MK2, MK2A, MK2B, MK2C...second marker, MK3C...third marker

Claims (13)

a control unit that detects the first information and the second information based on a captured image of a target marker on which a first marker indicating first information and a second marker indicating second information are superimposed,
the first marker is a marker that reflects electromagnetic waves in a predetermined first wavelength band and indicates the first information by a predetermined first type of encoding,
the second marker is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates the second information by a second type of encoding different from the first type,
the control unit detects the first information indicated by the first marker based on the captured image, and detects an outline of the first marker, and detects the second information indicated by the second marker based on the detected outline of the first marker and the captured image.
Information processing device. At least one of the electromagnetic waves in the first wavelength band and the electromagnetic waves in the second wavelength band is visible light,
The hue of the first marker is different from the hue of the second marker.
The information processing device according to claim 1 . one of the hue of the first marker and the hue of the second marker is a hue located at a first vertex among three vertices of an equilateral triangle inscribed in a predetermined hue circle,
the other of the hue of the first marker and the hue of the second marker is a hue located at a second vertex, which is different from the first vertex, among the three vertices in a predetermined hue circle;
The information processing device according to claim 2 . The first marker is an ArUco marker;
The information processing device according to claim 1 . The second marker is a QR code .
The information processing device according to claim 1 . The target marker is superimposed with one or more markers in addition to the first marker and the second marker.
The information processing device according to claim 1 . an imaging unit including a lens to which a filter that transmits at least one of the electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal;
The control unit detects the first information and the second information based on the captured image captured by the imaging unit.
The information processing device according to claim 1 . Further comprising a communication unit for communicating with a mobile object,
the moving body includes an imaging unit having a lens to which a filter that transmits at least one of the electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal;
The first information is information indicating a relative position and orientation between the imaging unit and the target marker,
The second information is information indicating a position and an orientation of the target marker in a predetermined coordinate system,
the control unit detects the first information and the second information based on the captured image captured by the imaging unit, calculates a position and an attitude of the moving body in the predetermined coordinate system based on the detected first information and the detected second information, and controls the moving body via the communication unit based on the calculated position and attitude of the moving body.
The information processing device according to claim 1 . The moving object is a drone or a robot.
The information processing device according to claim 8 . and after detecting the first information and the second information, the control unit stores, in a storage unit, history information that associates information indicating the first marker from which the first information was detected with the detected second information.
The information processing device according to claim 1 . The computer of the information processing device according to any one of claims 1 to 6 ,
a detection step of detecting the first information and the second information based on the captured image in which the target marker is captured;
A program for executing. A communication unit that communicates with a mobile object;
a control unit that detects the first information and the second information based on a captured image in which a first marker indicating first information and a second marker indicating second information are superimposed;
Equipped with
The first marker reflects electromagnetic waves in a predetermined first wavelength band and has a predetermined first type of code.
a marker which indicates the first information by encoding the first information;
The second marker reflects electromagnetic waves in a second wavelength band different from the first wavelength band, and
a marker indicating the second information by a second type of encoding different from the first type;
the moving body includes an imaging unit having a lens to which a filter that transmits at least one of the electromagnetic waves in the first wavelength band and the second wavelength band can be attached, and an imaging element that converts the electromagnetic waves focused by the lens into an electric signal;
The first information is information indicating a relative position and orientation between the imaging unit and the marker,
the second information is information indicating a position and an orientation of the marker in a predetermined coordinate system,
the control unit detects the first information and the second information based on the captured image captured by the imaging unit, calculates a position and an attitude of the moving body in the predetermined coordinate system based on the detected first information and the detected second information, and controls the moving body via the communication unit based on the calculated position and attitude of the moving body.
Information processing device. a control unit that detects the first information and the second information based on a captured image in which a first marker indicating the first information and a second marker indicating the second information are superimposed,
the first marker is a marker that reflects electromagnetic waves in a predetermined first wavelength band and indicates the first information by a predetermined first type of encoding,
the second marker is a marker that reflects electromagnetic waves in a second wavelength band different from the first wavelength band and indicates the second information by a second type of encoding different from the first type,
and after detecting the first information and the second information, the control unit stores, in a storage unit, history information that associates information indicating the first marker from which the first information was detected with the detected second information.
Information processing device.

Images