JP7821466B2 - Measurement system and measurement method - Google Patents

Description

The present invention relates to three-dimensional photometry using targets.

Various sensing technologies are being researched with the aim of improving the efficiency of agricultural work. One such technology is the use of three-dimensional photometry of crops. In three-dimensional photometry, targets are used to identify correspondences between stereo images and to orient the camera. For example, Patent Document 1 describes targets used for three-dimensional photometry in fields. Furthermore, Patent Documents 2 and 3 describe three-dimensional targets with multiple identification surfaces.

Japanese Patent Application Laid-Open No. 2021-139749 Japanese Patent Application Laid-Open No. 2002-354506 Japanese Patent Application Laid-Open No. 2003-42726

When taking 3D photographs in a field, it is not realistic to aim at each subject and take a photograph. A more practical method is to have a work robot equipped with a camera patrol the field and take photographs.

When using a mobile robot to photograph crops in a field, the camera attached to the mobile robot must be able to capture images of numerous identification targets placed in the field in an identifiable state. Incidentally, the identification targets used for orientation display a graphic used to detect the target's center in the image, as well as a graphic that serves as identification information. These identification targets should be photographed as closely as possible to the front. This is because photographing them from an angle distorts the graphic, reducing the accuracy of reading the identification code and the center position.

It is therefore important to point the mobile robot's camera directly at the target to be identified when taking photographs. On the other hand, when considering the creation of a 3D model based on stereophotography, it is necessary to take photographs from many different viewpoints. Therefore, it is necessary to photograph the target to be identified from different viewpoints and from the front.

In addition, there is a demand for identification targets that can be identified using a variety of methods, rather than being limited to a single identification method.

Against this background, the present invention aims to provide technology related to highly versatile identification targets.

The present invention provides a measurement system comprising a plurality of identification targets arranged in a field , a mobile body movable within the field, and a first camera and a second camera mounted on the mobile body and facing in different directions, wherein the plurality of identification targets face in different directions and each have a first surface and a second surface on which an identification code is displayed , the first camera being capable of facing directly on each of the first surfaces of the plurality of identification targets whose normal direction is the first direction, and the second camera being capable of facing directly on each of the second surfaces of the plurality of identification targets whose normal direction is a second direction different from the first direction, and while the mobile body is moving, the first camera being capable of intermittently photographing each of the first surfaces of the plurality of identification targets arranged at intervals in the direction of movement of the mobile body, and the second camera being capable of intermittently photographing each of the second surfaces of the plurality of identification targets arranged at intervals in the direction of movement of the mobile body .

In the present invention, the same identification code may be displayed on the first surface and the second surface.

In one embodiment of the present invention, the identification target has a polyhedral structure, the first surface is one surface of the polyhedron, and the second surface is another surface of the polyhedron.

In one embodiment of the present invention, the identification target has a truncated pyramid shape, and the first and second surfaces, as well as another surface bearing an identification code, are formed using the multiple inclined surfaces that make up the truncated pyramid and the smaller of the two parallel surfaces that make up the truncated pyramid.

In the present invention, an example is where, in a first state in which the first camera is capturing the first surface of one of the multiple identification targets in a frontal view, the second camera is not capturing the second surface of any of the multiple identification targets in a frontal view, and by moving the moving body from the first state, a second state is reached in which the first camera is not capturing the first surface of any of the multiple identification targets in a frontal view, and the second camera is capturing the second surface of any of the multiple identification targets in a frontal view.

In the present invention , the multiple identification targets may include targets of different sizes and/or shapes, and the differences in size and/or shape of the identification targets may be detected by a means for measuring three-dimensional information, thereby identifying the identification targets.

The present invention is an identification target that has a three-dimensional structure on which an identification code is displayed, the size and/or shape of the three-dimensional structure corresponds to the identification information of the identification code, and the identification information can be detected by the identification code and by the size and/or shape of the three-dimensional structure.

The present invention is a measurement method that uses a plurality of identification targets arranged in a field , and a first camera and a second camera that are mounted on a mobile body that can move within the field and face in different directions , wherein the plurality of identification targets face in different directions and each have a first surface and a second surface on which an identification code is displayed , the first camera is capable of facing directly on each of the first surfaces of the plurality of identification targets whose normal direction is the first direction, and the second camera is capable of facing directly on each of the second surfaces of the plurality of identification targets whose normal direction is a second direction different from the first direction, and while the mobile body is moving, the first camera is used to intermittently photograph each of the first surfaces of the plurality of identification targets that are arranged at intervals in the direction of movement of the mobile body, and the second camera is used to intermittently photograph each of the second surfaces of the plurality of identification targets that are arranged at intervals in the direction of movement of the mobile body .

The present invention provides technology for identifying targets with high versatility.

1A and 1B are conceptual diagrams of an embodiment. 1A and 1B are perspective views of an identification target. 1A and 1B are diagrams showing an example of a code display for identification purposes. 10 is a flowchart illustrating an example of a processing procedure. 10A, 10B, and 10C are perspective views showing other examples of the identification target.

1. First embodiment (configuration)
Figure 1 shows a measurement system deployed in a farm field 100. Figure 1(A) is a front view of a work robot (mobile robot) 110, and Figure 1(B) is a side view of the work robot 110.

Shelves 101 are provided in the field 100, and the shelves 101 are used to cultivate crops 102 (e.g., grapes). The crops cultivated are not limited to grapes, and may be fruits or vegetables other than grapes.

A work robot 110, a mobile body equipped with a camera, moves across the ground 120 of the farm field 100. The work robot 110 is equipped with a drive means such as a motor and moves autonomously using wheels. The work robot 110 moves along a predetermined path. Of course, it is also possible for the work robot 110 to be configured to autonomously set its own movement path. It is also possible for the robot to move using caterpillars instead of wheels, or rails or wires. A UAV can also be used as a mobile body equipped with a camera.

The work robot 110 is equipped with multiple cameras. In Figure 1, the work robot 110 is equipped with camera 111 facing diagonally upward at 45 degrees in front of the vehicle 110, camera 112 facing vertically upward, camera 113 facing diagonally upward at 45 degrees behind the vehicle, camera 114 facing diagonally upward at 45 degrees to the left in the direction of travel (positive direction of the X-axis), and camera 115 facing diagonally upward at 45 degrees to the right in the direction of travel (positive direction of the X-axis).

In this example, each camera repeatedly captures still images. The interval between captures is set so that, when capturing images while the work robot 110 is moving, the captured images overlap between the nth capture and the n+1th capture. When capturing video, overlapping images are cropped and used as the captured image.

Figure 1 shows identification targets 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, and 131 arranged on shelf 101. Each identification target has a different identification marking, but they all have the same shape.

The position of at least part of the identified target is known. The coordinate system used to describe the position may be an absolute coordinate system or a local coordinate system. An absolute coordinate system is the coordinate system used in GNSS and maps.

In this example, three rows of identification targets are arranged extending in the X-axis direction along the movement path of the work robot 110. The central row of these three rows of identification targets is made up of identification targets 121, 122, 123, 124, 125, 126, 127, 128, and 129 shown in Figure 1(B). Similarly, the row of identification targets extending in the X-axis direction to which identification target 130 belongs, and the row of identification targets extending in the X-axis direction to which identification target 131 belongs are also arranged.

In Figure 1, all of the identification targets have the same shape, but the identification data displayed varies. The shapes of the identification targets are explained below. Here, identification target 122 will be explained as a representative example.

Figure 2 shows the identification target 122. Figures 2(A) and 2(B) show the target from different perspectives.

The identification target 122 has the shape of an upside-down truncated quadrangular pyramid. Figure 2 shows the inclined surfaces 122A, 122B, 122C, 122D, and the bottom surface 122E of the identification target 122. The inclined surfaces 122A, 122B, 122C, 122D, and the bottom surface 122E serve as code display surfaces on which the same identification code is displayed.

Sloped surface 122A is a plane facing diagonally downward at 45 degrees to the positive X-axis. Sloped surface 122C is the slope on the opposite side of slope 122A in the X-axis direction, and is a plane facing diagonally downward at 45 degrees to the negative X-axis.

Sloped surface 122B is a plane facing diagonally downward at 45° in the negative direction of the Y axis. Sloped surface 122D is the slope on the opposite side of slope 122B in the Y axis direction, and is a plane facing diagonally downward at 45° in the positive direction of the Y axis. Bottom surface 122E is a plane facing vertically downward (negative direction of the Z axis).

Sloped surface 122C is oriented so that it can face camera 111 directly. That is, in FIG. 1, camera 111 is oriented so that its optical axis faces diagonally upward at 45 degrees along the positive X-axis. In the state shown in FIG. 1(B), camera 111 and slope 122C face each other directly, i.e., slope 122C is oriented diagonally downward at 45 degrees along the negative X-axis so that camera 111 photographs slope 122C from the front. This is also true for the corresponding slopes of identification targets 121, 123-129.

Using a similar concept, inclined surface 122A is oriented so that it can face camera 113 directly. That is, in Figure 1, camera 113 is oriented so that its optical axis faces upward at an angle of 45 degrees in the negative direction of the X-axis. In the state shown in Figure 1(B), camera 113 and inclined surface 122A do not face each other directly, but by moving work robot 110 in the positive direction of the X-axis, a situation is created in which camera 113 can photograph inclined surface 112 from the front. To achieve this situation, inclined surface 122A is oriented downward at an angle of 45 degrees in the positive direction of the X-axis. The same applies to the inclined surfaces corresponding to identification targets 121, 123-129.

The orientation of the lower surface 122E is set so that it faces directly toward the camera 112, which is facing vertically upward, when the camera 112 is positioned vertically downward. This is also true for the lower surfaces corresponding to the identification targets 121, 123-129.

The slope of identification target 130 facing diagonally downward at 45° in the negative Y-axis direction is oriented so that it can directly face camera 114 facing diagonally upward at 45° in the positive Y-axis direction. This is also true for other identification targets (not shown) lined up in the X-axis direction and included in the row of identification target 130. The slope of identification target 131 facing diagonally downward at 45° in the positive Y-axis direction is positioned so that it can directly face camera 115 facing diagonally upward at 45° in the negative Y-axis direction. This is also true for other identification targets (not shown) lined up in the X-axis direction and included in the row of identification target 131.

In the above explanation, it has been explained that the orientation of the code display surface of each identification target is set to match the orientation of the camera, but it can also be thought of as the camera being set to match the orientation of the code display surface. Alternatively, it can be thought of as the orientation of the code display surface and camera being set so that the above orientation relationship is achieved. Furthermore, by attaching a code to the top surface of this inverted trapezoid target, it can also be recognized from the air by UAVs, etc.

The same code display is placed on each of the five code display surfaces of each identification target. Figure 3(A) shows an example of a code display. In this code display, the center of the code display in the image is recognized by detecting the center of the central circle and the center of curvature of the arc in a photographed image of the code display, and the position and length of the arc are detected, allowing the identification information indicated by the code display to be recognized.

Identification targets 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, and 131 have code display surfaces facing five different directions, allowing the code display surfaces to be photographed from the front from five different directions. In other words, they are targets that are easy to identify from multiple directions.

As mentioned above, if the code display in Figure 3 is photographed from an oblique angle, the shape of the figure will be distorted, increasing the possibility of erroneous detection of the center position or problems reading the identification information. Since the identification targets 121-131 can be viewed directly from five directions, they can be easily identified from multiple directions.

Furthermore, in the system of Figure 1, any of the cameras facing in various directions can photograph the code display surface of the identification target from the front or a position close to the front. For example, in the state shown in Figure 1(B), one sloping surface of identification target 122 faces directly toward camera 111. Furthermore, the bottom surface of identification target 125 faces directly toward camera 112. Furthermore, one sloping surface of identification target 128 faces directly toward camera 113.

Here, when the work robot 110 moves in the positive direction of the X-axis, it deviates from the above-mentioned facing state. However, with further movement, one slope of the identification target 121 faces directly toward the camera 111, the bottom surface of the identification target 124 faces directly toward the camera 112, and one slope of the identification target 127 faces directly toward the camera 113.

In this way, while the work robot 110 is moving, the cameras 111, 112, and 113 intermittently, though not always, face the code display surface of the identification target. Even if they do not face it completely directly, a situation as close to a direct face as possible can be created for as long as possible.

The same applies to cameras 114 and 115. Similar to identification targets 121, 122, 123, 124, 125, 126, 127, 128, and 129, multiple identification targets are also arranged in the X-axis direction of identification targets 130 and 131.

Now, let's focus on the identification target 130. In this case, when the work robot 110 moves in the positive X-axis direction from a state in which the camera 114 is directly facing one of the slopes of the identification target 130, the identification target 130 moves out of view of the camera 114, but soon the next identification target will be visible in front of the camera 114. The same is true for the camera 115.

Furthermore, when focusing on the identification target 122, the code display surface 122D can face the camera 111, and the code display surface 122A can face the camera 113. Then, when the work robot 110 moves in the positive direction of the X axis from the state shown in Figure 1(B) where the code display surface 122D faces the camera 111, the code display surface 122A faces the camera 113.

In this way, while the work robot 110 is moving, each camera can capture images of the identification target from the front as often as possible.

There is no limit to the number of cameras, but the more cameras you have, the more thorough the capture of images. The same applies to identifying targets.

One method for moving the cameras is to use a manually-moved cart. It is also possible for workers to carry multiple cameras and take pictures while they move. In this case, multiple cameras are fixed to the worker's head (e.g., a helmet) or work vest, facing in multiple directions, and the worker can simultaneously take pictures in multiple directions as he or she moves.

(Example of processing)
An example of a procedure for obtaining identification information of an identification target from a photographed image will now be described with reference to a flowchart of FIG.

First, photographs are taken within the field (step S101). For example, the work robot 110 in Figure 1 is moved thoroughly within the field, taking photographs using the multiple cameras it is equipped with. When focusing on a particular camera, the movement speed and photographing interval of the work robot 110 are adjusted so that the image taken at a first timing and the image taken at the following second timing partially overlap. The images taken at the first timing and the second timing become the stereo images used for stereophotography.

Next, image data for the multiple images captured in step S101 is received by an image processing device (step S102). The image processing device is configured, for example, as a PC (personal computer), and performs processing related to the detection of identification targets from the captured images (image recognition) and sfm (Structure from Motion). sfm processing includes the extraction of feature points from the captured images, identification of correspondences between multiple images that make up a stereo image, orientation using identification targets, generation of point cloud data related to objects captured in the captured images, and creation of a three-dimensional model based on the point cloud data.

The image processing device that receives the image data performs the following process. First, it detects the code display (see Figure 3, for example) displayed on the code display surface of the identification target from each captured image (step S103). If multiple different code displays are captured in a single captured image, each code display is detected.

When multiple identical code indications are captured in a single captured image, the reliability of code detection is improved. For example, suppose a first code indication and a second code indication with the same content are detected, but some of the different information is missing from each. In this case, by interpolating the missing information, more reliable detection of the code content is possible. It is also possible to reduce false detection of the code content.

Furthermore, priority is given to detecting code displays photographed from the front as much as possible. Whether or not the image was photographed from the front is determined by evaluating the distortion of the image. For example, in the case of Figure 3(A), if the image was not photographed from the front, the arc will be distorted and will become a curved line rather than an arc. By selecting a code display that is closer to an arc, it is possible to select a code display photographed from a closer front. For example, in the case of Figure 3(B), if the image was not photographed from the front, there will be deviation from the rectangular shape. By selecting a code display that is closer to a rectangle, it is possible to select a code display photographed from a closer front.

In addition, in the case of Figure 3(A), if the image is not taken from the front, there will be inconsistencies in the positions of the centers of curvature of the multiple arcs, and inconsistencies between the positions of the centers of curvature of the arcs and the center positions of the circles. One way to select a code display that is taken from a more frontal angle is to select a code display with the least degree of inconsistency.

Alternatively, it is possible to evaluate the reliability of targets where multiple identical codes are detected and register them as points on a map, or if the target has already been registered as a point on a map, use those coordinates as the coordinates of the detected target.

Once a code display is detected, the code content (center position and identification information) is obtained (step S104). Next, feature points are extracted from each captured image (step S105), and the correspondence between the stereo images is identified (step S106).

Next, orientation is performed using the code displayed in the stereo images to determine the camera position and orientation (step S107). Next, the three-dimensional positions of feature points in the stereo images are calculated using the forward intersection method to obtain point cloud data (step S108). Next, a three-dimensional model of the subject is created based on the point cloud data obtained in step S108 (step S109).

In the above process, there is a high probability that many of the images will be taken from the front of the code display, which helps reduce false or incomplete detection of the code display.

The above example shows stereo measurement, but it is also possible to use markers such as those shown in Figure 3(B) to perform measurements from a single image using single-photo orientation. In this case, single-photo orientation can be performed by detecting multiple feature points on the target and providing the condition that the target is placed on a flat surface. In this case, in order to utilize the flat surface condition, it is even more important that the targets are facing directly than with stereo.

Figure 3 (B) shows an example of an Ar marker as a code representation. An Ar marker is composed of multiple rectangular shapes. The corners of each rectangular shape are extracted from the captured image as feature points. The positions and combinations of the rectangular shapes are predetermined, and identification information is read from them. In addition, the positional relationships of the corners of each rectangular shape are known, and by providing the condition that they are arranged on a plane, the position and orientation of the camera can be calculated using single-photo orientation.

Specifically, the position and orientation of the camera that captured the code display surface relative to the code display surface are calculated using the backward intersection method, with the corners of the rectangular shape set as known points.

2. Second embodiment In addition to a camera facing the code display, multiple cameras may be placed with their optical axes slightly offset from the optical axis of the camera. Although the camera with the slightly offset optical axis does not face the code display of the identification target exactly, it can capture many images that are close to being directly facing the code display.

In this case, even if the positional relationship between the moving object and the identification target is somewhat approximate, one of the multiple cameras that are oriented similarly will be directly facing or nearly facing the identification surface (code display surface) of the identification target. The ground in a field is not necessarily horizontal or flat, and it is possible that the moving object carrying the camera may be tilted. Furthermore, when a camera is attached to a person, the direction of the camera's optical axis will not be constant. In such cases, by arranging multiple cameras with slightly offset optical axis directions, one of them will be directly facing (or nearly facing) the identification surface. This makes it possible to obtain a captured image that is directly facing the identification surface with a high probability.

It is also possible to use a stereo camera using two or more cameras as cameras 111 to 115 in Figure 1.

3. Third Embodiment Different identification codes can be used on the multiple code display surfaces of an identification target. For example, different codes can be displayed on all or multiple of the inclined surfaces 122A, 122B, 122C, and 122D and the bottom surface 122E in FIG. 2 . This makes it possible to determine the direction of a moving object relative to the target. This allows for more reliable navigation and map creation when a map is not available or when creating a map.

4. Fourth Embodiment The orientation of the camera and the orientation of the identification target may be set so that the code display surface faces directly in the upper right direction and/or the code display surface faces directly in the upper left direction when viewed in the direction of travel of the mobile body (work robot 110) on which the camera is mounted.

5. Fifth Embodiment Examples of other shapes of identification targets are shown in Figure 5. Figure 5(A) shows an identification target 400 having code display surfaces 401, 402, 403, 404, and 405. The code display surfaces 401 to 405 are set to face in different directions, for example, at intervals of 30°.

Figure 5(B) shows an identification target 410 with code display surfaces 411 and 412. Figure 5(C) shows an identification target 420 with code display surfaces 421, 422, and 423. Note that there is another code display surface of the same shape on the back side of code display surface 422, but it is hidden and not visible in the figure.

6. Sixth Embodiment In addition to the form in which identification is performed by capturing an image, it is also possible to use a means capable of acquiring three-dimensional information of an object, such as a Laider or a depth camera, to identify an identification target.

Means for obtaining three-dimensional information about an object include stereo cameras, Laiders, ToF cameras, cameras that obtain depth information using projected light such as dot patterns, radar, and combinations of two or more of these.

These identification targets can take several forms. The first form is one in which the identification is based on differences in size and/or shape of each identification target. For example, Laider can evaluate the size and shape of the measurement target, so it can detect differences in size and shape of the identification target.

For example, by preparing different shapes corresponding to identification codes, it becomes possible to use camera images in conjunction with sensors that capture three-dimensional shapes, such as Laider, which can greatly improve the efficiency of the process of unifying the three-dimensional space captured by each sensor and the three-dimensional space captured by images on the same map.

In the second form, the truncated pyramidal identification target shown in Figure 2 is combined with a rectangular parallelepiped, and identification is performed based on the difference in the length of the rectangular parallelepiped (dimension in the Z-axis direction).

The third method is to form the code display as a protruding convex or recessed (groove) shape, and then detect the displayed content by measuring the uneven shape using a laser scan.

The fourth method is to form the code marking and the non-coded portion from materials with significantly different absorbance (or reflectance) for the wavelength of the laser light used by the Laider. In this case, the code marking can be recognized based on the intensity of the reflected light of the laser light used for measurement. It is also possible to combine two or more of the first through fourth methods.

A specific example is shown below. As a simple example, let's assume that a square, pentagonal, and hexagonal pyramid are used as identification targets, with three different sizes. In this case, by combining three different shapes and three different sizes, nine different codes (identification information) can be handled.

In this case, the code content on the code display surface of each identification target is set to correspond to the nine types of code information mentioned above. Note that the display content on the multiple code surfaces of each identification target is the same. The display format uses the codes shown in Figure 3(A) or Figure 3(B), for example.

For example, suppose the code content is the numbers 1 to 9. In this case, the first identification target has a shape and size corresponding to the number 1 and is equipped with a code display corresponding to the number 1. The second identification target has a shape and size corresponding to the number 2 and is equipped with a code display corresponding to the number 2. The same applies to the third to ninth identification targets.

In this case, the code can be detected both from the image captured by the camera and based on the size and shape of the object using a means of acquiring three-dimensional information, such as a lidar or depth camera. For example, in the above example, target number 1 is identified by capturing the image with the camera, and target number 1 is identified by detecting the size and shape corresponding to number 1 using a lidar or depth camera. In other words, the same target can be identified using different observation (measurement) means.

This makes it possible to identify common identification targets in both sets of data, even when the data is measured using different measurement methods, such as image data captured using a moving camera and Lidar measurement data, and easily determine the relationship between the two sets of data.

In addition, for example, it is easy to integrate a 3D model based on sfm and a 3D model based on lidar, or to integrate 3D data obtained by a depth camera into a 3D model based on sfm and an image captured by the work robot 110.

For example, suppose there is a 3D model (first 3D model) of a farm field created using sfm based on images captured by the work robot 110. On the other hand, suppose there is a 3D model (second 3D model) of a portion of the farm field measured using lidar.

Here, when integrating the first 3D model and the second 3D model, it is necessary to identify the correspondence between the first 3D model and the second 3D model, or to align the first 3D model and the second 3D model.

This task requires the task of finding (searching for) common parts between both 3D models. There are various methods for searching for common parts between 3D models, but they have problems with increased computational effort and false matching, making them impractical.

When the above-mentioned identification target is used, it is possible to know in which of the captured images used to create the first 3D model the identification target detected in the second 3D model appears. This makes it easy to know which part of the first 3D model the second 3D model corresponds to.

If you can determine which part of the first 3D model the second 3D model corresponds to, you can easily integrate the first and second 3D models.

A specific example will be described below. For example, suppose that photography is performed using the work robot 110 shown in Figure 1, and a 3D model of the farm field 100 (a 3D model of the entire farm field) is created using sfm (see, for example, the processing in Figure 4). On the other hand, suppose that measurements are performed using Lidar on the fruits of some crops in the farm field 100, and a 3D model of the fruits (a 3D model of the fruits) is obtained.

Here, suppose that an identification target of this embodiment is located near the fruit that is the subject of Lidar measurement, and that at least one identification target of this embodiment is included in the laser scan data (laser scan point cloud) that forms the basis of the "3D model of the fruit." In this case, along with the "3D model of the fruit," a 3D model of the identification target located nearby is also obtained. In other words, the data for the "3D model of the fruit" also includes the 3D model of the identification target.

In this case, code information (identification information) based on the size and/or shape of the identification target is extracted from the laser scan data that forms the basis of the "real 3D model." Let's say the extracted code information for the identification target is "code number 5," for example.

In this case, a search is made for an image that captures the identification target "Code Number 5" from the group of captured images that form the basis of the 3D model of the entire field. This identifies the correspondence between the "fruit 3D model" obtained through Lidar measurement and the image captured by the camera mounted on the work robot 110. Normally, there will be multiple images that capture this "Code Number 5." This is because the capture conditions for the images captured by the camera mounted on the work robot 110 are set so that multiple images with overlapping capture ranges can be obtained in order to perform SFM.

Here, the 3D model created by sfm based on the multiple captured images for which the above correspondence has been identified includes a 3D model of the identification target with "code number 5." Therefore, the 3D model of the identification target with "code number 5" can be identified in both the "3D model of the field" and the "3D model of the fruit."

For example, by overlapping the 3D model portion of the identification target "Code No. 5" in the "3D model of the field" with the 3D model portion of the identification target "Code No. 5" in the "3D model of the fruit," rotating one relative to the other and translating it if necessary, and then zooming in and out, the state in which the two 3D models related to the identification targets overlap is searched for.

This search identifies the correspondence between the "3D model of the field" and the "3D model of the fruit." In other words, it identifies which parts of the "3D model of the field" correspond to the "3D model of the fruit." Then, by identifying the correspondence between the two 3D models, the "3D model of the field" and the "3D model of the fruit" can be integrated.

At this time, the exterior orientation elements (position and orientation) of the Lidar in the coordinate system describing the "3D model of the field" are also determined.

It is also possible for the identification target to be distinguishable by using only one of differences in size and shape.

1 shows a configuration in which the identification target is arranged facing downward, but it is also possible to arrange it facing horizontally (for example, by arranging the code display surface 122E in FIG. 2 facing horizontally) or facing vertically upward (upside down from FIG. 2). In this case, the direction of the optical axis of the camera is set according to the orientation of the code display surface.

8. Eighth Embodiment The structure of the identification target may be a polygonal truncated pyramid (n-sided truncated pyramid) such as a triangular truncated pyramid, a pentagonal truncated pyramid, or a hexagonal truncated pyramid, in addition to a quadrangular truncated pyramid.

8. Eighth Embodiment In another example, these identification targets can be placed on the ground. By placing them on the ground, it becomes possible to handle, for example, outdoor-grown cabbage, lettuce, strawberries, and other plants. In this case, by placing a camera on the moving body facing downward or diagonally downward instead of upward, it becomes possible to achieve the same results as in the first embodiment.

9. Ninth Embodiment If the identification target 122 in FIG. 2 is placed upside down, it becomes a target suitable for shooting from a mobile object moving through the air. In this case, identification from various directions is also possible. Examples of mobile objects in this case include UAVs, robots that move through the air using wires, and robots that move on rails placed in high places. In this case, multiple cameras facing various directions, including downward, are placed on the mobile object moving through the air.

10. Tenth Embodiment It is also possible to prepare two identification targets 122 as shown in Figure 2 and join the wider parallel surfaces together to form an identification target. In this case, identification is possible from both below and above. In this case, for example, an image is taken from below by a work robot traveling on the ground as shown in Figure 1, and an image is taken from above by a UAV. If this identification target is used by laying it on its side, identification is possible from both the left and right sides.

100...field, 101...shelf, 102...crop, 111...camera, 112...camera, 113...camera, 114...camera, 115...camera, 120...ground, 121...identification target, 121A...code display surface, 121B...code display surface, 121C...code display surface, 121D...code display surface, 121E...code display surface, 122...identification target, 123...identification target, 124...identification target, 125...identification target, 126...identification target, 127...identification target, 128...identification target, 129...identification target, 130...identification target, 131...identification target.

Claims (7)

a plurality of identification targets arranged in a field ;
a mobile body that can move within the farm field;
a first camera and a second camera provided on the moving body and facing in different directions;
Equipped with
the plurality of identification targets each have a first surface and a second surface facing in different directions and displaying an identification code ;
the first camera can face directly onto each of the first surfaces of the plurality of identification targets, the normal direction of which is a first direction, and the second camera can face directly onto each of the second surfaces of the plurality of identification targets, the normal direction of which is a second direction different from the first direction;
A measurement system in which, while the moving body is moving, the first camera is capable of intermittently photographing each of the first surfaces of a plurality of the identification targets arranged at intervals in the direction of movement of the moving body, and the second camera is capable of intermittently photographing each of the second surfaces of the plurality of the identification targets arranged at intervals in the direction of movement of the moving body . The measurement system according to claim 1 , wherein the first surface and the second surface have the same identification code displayed thereon. the identification target has a polyhedral structure;
the first face is one face of the polyhedron,
The measurement system according to claim 1 , wherein the second surface is another surface of the polyhedron. the identification target has a truncated pyramid shape;
The measurement system of claim 3, wherein the first surface, the second surface, and another surface having an identification code are formed using the multiple inclined surfaces that form the truncated pyramid and the smaller surface of the two parallel surfaces that form the truncated pyramid. In a first state in which the first camera captures the first surface of any of the plurality of identification targets in a frontal direction, the second camera does not capture any of the second surfaces of the plurality of identification targets in a frontal direction;
A measurement system as described in any one of claims 1 to 4, wherein by moving the moving body from the first state, a second state is reached in which the first camera does not capture any of the first surfaces of the multiple identification targets directly in front, and the second camera captures the second surface of one of the multiple identification targets directly in front. The plurality of identification targets include targets of different sizes and/or shapes;
A measurement system according to any one of claims 1 to 5, wherein the difference in size and/or shape of the identification target is detected by a means for measuring three-dimensional information, thereby identifying the identification target. A measurement method using a plurality of identification targets arranged in a farm field , and a first camera and a second camera provided on a mobile object movable in the farm field and facing in different directions, comprising :
the plurality of identification targets each have a first surface and a second surface facing in different directions and displaying an identification code ;
the first camera can face directly onto each of the first surfaces of the plurality of identification targets, the normal direction of which is a first direction, and the second camera can face directly onto each of the second surfaces of the plurality of identification targets, the normal direction of which is a second direction different from the first direction;
A measurement method in which, while the moving body is moving, the first camera is used to intermittently photograph each of the first surfaces of a plurality of the identification targets arranged at intervals in the direction of movement of the moving body, and the second camera is used to intermittently photograph each of the second surfaces of the plurality of the identification targets arranged at intervals in the direction of movement of the moving body .