JP7523940B2 - Photographing device and information processing device - Google Patents

Description

The present invention relates to a photographing device and an information processing device that photographs while moving.

Patent document 1 describes detecting the direction of a work vehicle, and when the direction of the vehicle body detected by the direction detection member differs from the set travel route by a predetermined value or more, the steering member is steered in a direction toward the set travel route.

JP 2019-115309 A

It is possible to analyze images of farm fields to plan yields or deal with pests and diseases. However, the technology described in Patent Document 1 does not take into consideration the analysis of farm field conditions through image processing.

Therefore, in order to solve the above problems, the present invention aims to provide an imaging device and an information processing device that can acquire information for analyzing the condition of the place where the vehicle has traveled, such as a farm field.

The photographing device of the present invention is a photographing device that performs photographing processing while traveling, and includes an image generating photographing unit that photographs a plurality of objects arranged in a plurality of rows in the direction of movement of the autonomous driving device while the autonomous driving device is moving, a storage unit that stores images including the photographed plurality of objects, a selection unit that selects a plurality of images photographed as the autonomous driving device moves, and an image processing unit that concatenates the selected plurality of images.

According to this invention, images of the part traveled by the camera can be selected according to the travel distance, and then linked to generate a single image. This makes it easy for the administrator to grasp the surface.

This invention makes it easy to link images of the areas traveled by the imaging device.

1 is a schematic diagram of an automatic driving device 100 according to an embodiment of the present disclosure traveling through a farm field H. FIG. 1 is a block diagram showing a functional configuration of an automatic driving device 100 according to an embodiment of the present disclosure. 1 is a diagram showing the relationship between the automatic driving device 100 traveling in a field H and image data captured by a camera 117. FIG. 1 is a diagram showing a schematic diagram of a positional relationship between a photographing location in a farm field H and image data. FIG. FIG. 13 is a diagram illustrating a process of obtaining a recognition window for a crop S and using the same to perform synthesis. 11A and 11B are diagrams illustrating a synthesis process when a crop S is photographed at an angle in image data. 4 is a flowchart showing the operation of the automatic driving device 100. FIG. 11 is a graph showing behavior information over time. FIG. 11 is a diagram showing a functional configuration of an automatic driving device 100a in a modified example. 13 is an explanatory diagram showing a procedure for combining image data in a modified example. FIG. 2 is a diagram illustrating an example of a hardware configuration of a control portion of an automatic driving device 100 according to an embodiment of the present disclosure. FIG.

Embodiments of the present disclosure will be described with reference to the accompanying drawings. Where possible, identical parts will be designated by the same reference numerals, and duplicate explanations will be omitted.

Figure 1 is a schematic diagram of an automated driving device 100 according to an embodiment of the present disclosure traveling through a field H. As shown in Figure 1, the automated driving device 100 recognizes the arrangement of crops S planted in the field H, and travels along a driving route R that follows that arrangement. The automated driving device 100 makes a U-turn at the end of the field H and travels through the entire field H.

The automated driving device 100 is equipped with a camera 101, and recognizes the arrangement of the crops S based on image data captured by the camera 101. The automated driving device 100 is equipped with a wheel drive mechanism, a steering mechanism 106, and a working unit 107. The working unit 107 is, for example, a weeding mechanism. The automated driving device 100 weeds other than the crops S by traveling through the field H. The weeding mechanism is one example, and the working unit 107 may be configured to perform other working mechanisms.

FIG. 2 is a block diagram showing the functional configuration of the autonomous driving device 100 in one embodiment of the present disclosure. As shown in the figure, the autonomous driving device 100 includes a camera 101, a recognition unit 102, a perspective transformation unit 103, an adjustment unit 104, a speed control unit 105, a wheel drive mechanism and a steering mechanism 106, a working unit 107, a behavior detection unit 111, a still image storage unit 112, a selection unit 113, an image processing unit 115, a composite image storage unit 116, and a camera 117.

Camera 101 is positioned facing the direction of travel of automatic driving device 100 and is a part that photographs crops and the like arranged in the direction of travel and acquires image data for driving control. Camera 101 may capture video and extract image data from it, or may capture still images at a predetermined interval.

The recognition unit 102 is a part that recognizes the crop S, which is the detection target, from the image data captured by the camera 101. The recognition unit 102 recognizes the type (or weeds), coordinates, and size of the crop S from the image data. In this disclosure, the recognition unit 102 is configured to recognize the crop S but not weeds.

The perspective transformation unit 103 is a part that performs perspective transformation processing on the image data acquired by the camera 101. The perspective transformation unit 103 is a part that performs coordinate transformation, and converts the image data of the crop captured from the front by the camera 101 into image data viewed from a viewpoint above the center of the capture range.

The adjustment unit 104 is a part that controls the wheel drive mechanism and steering mechanism 106 so that the vehicle moves along the crop rows in the field converted by the perspective conversion unit 103. The adjustment unit 104 recognizes the direction of the crop rows based on the crop rows recognized by the recognition unit 102, and determines the direction of movement.

The speed control unit 105 adjusts the speed when making a U-turn at the edge of the field H, and controls the wheel drive mechanism and the steering mechanism 106. The adjustment unit 104 recognizes the edge of the field H based on the crop row recognized by the recognition unit 102, and determines the position to make a U-turn.

The wheel drive mechanism and steering mechanism 106 are mechanical parts of the automated driving device 100. The wheel drive mechanism is a mechanism for moving at a speed controlled by the adjustment unit 104. The steering mechanism 106 is a mechanism for moving in a direction that follows the direction of travel set by the adjustment unit 104. The automated driving device 100 has other mechanisms, but they will not be described here.

The working unit 107 is a part that performs weeding processing as the automatic driving device 100 moves. The working unit 107 is equipped with a claw-shaped tool, and performs weeding by dragging the claw-shaped tool while the automatic driving device 100 is moving. However, it is not limited to this, and may also be a mechanism for top dressing and spraying pesticides. It may also be equipped with a robot arm and be used to thin out and harvest crops.

The behavior detection unit 111 is a part that detects the behavior of the automated driving device 100 while it is driving, and is composed of, for example, a gyro sensor and an acceleration sensor. The behavior detection unit 111 detects the instantaneous acceleration of the automated driving device 100, as well as other parameters such as the yaw angle, pitch angle, and roll angle, to detect its behavior (attitude, inclination, etc.).

The still image storage unit 112 is a part that stores still images captured by the camera 117 in association with behavior information detected by the behavior detection unit 111. The still image storage unit 112 also stores driving information (distance, time, etc.) from the wheel drive mechanism and steering mechanism 106, and associates it with image data.

The selection unit 113 is a part that selects one still image determined according to the driving state of the automatic driving device 100 from among the multiple still images stored in the still image storage unit 112. For example, the selection unit 113 selects image data that includes a part of the crop S included in the image data selected immediately before from among multiple image data including the crop S of multiple crop rows photographed by the camera 117. The immediately before image data and the selected image data need to be linked by positioning the crop S. Therefore, the selection unit 113 selects image data such that the crop S lined up behind the immediately before image data in the traveling direction and the crop S lined up in the traveling direction of the selected image data are the same crop S. The selection unit 113 selects multiple image data by repeating this process. Note that the selection unit 113 does not determine that the crops S are the same, but selects the same crop S based on driving information such as the distance (time or other factors such as the number of wheel rotations) traveled by the automatic driving device 100. This selection process may be performed during automatic driving, or may be performed all at once after the automatic driving is completed. When performing image selection processing during autonomous driving, image accumulation can be kept to a minimum.

The image processing unit 115 is a part that generates composite image data by linking multiple image data selected by the selection unit 113. That is, the image processing unit 115 recognizes one or multiple crops S included in the selected image data, obtains the recognition area of the crop S, and links the multiple image data based on the recognition area included in the multiple image data. That is, when linking image data captured earlier and later in the chronological order, the linking is performed based on the recognition area of the crop S.

The composite image storage unit 116 is a part that stores the composite image data generated by linking the images by the image processing unit 115.

The camera 117 is a part that photographs the crops S in the field H to obtain still images, and photographs the crops S at predetermined intervals or after a predetermined distance is traveled while traveling. The predetermined range photographed is shifted little by little depending on the travel. The camera 117 may take multiple still images that include the same crops S arranged in an array. The selection unit 113 selects the still image with the best image quality from the multiple still images. The camera 117 is positioned at the rear of the automatic traveling device 100 in an orientation that allows it to photograph the crops S from above.

Next, the linking process of the image data captured by the camera 117 in the image processing unit 115 will be described. FIG. 3 is a diagram showing the relationship between the automated driving device 100 traveling in the field H and the image data captured by the camera 117. As shown in FIG. 3, the camera 117 acquires image data G1 to G3 through a photographing process according to each of the positions P1 to P3 of the automated driving device 100.

Figure 4 is a diagram showing a schematic diagram of the positional relationship between the photographing locations in field H and the image data. As shown in the diagram, part H11 in field H corresponds to image data G11, part H12 in field H corresponds to image data G12, and part H13 in field H corresponds to image data G13.

The still image storage unit 112 stores multiple image data G, and the selection unit 113 selects one image data G for each distance (or other driving information such as time) traveled by the automated driving device 100. In FIG. 4, the selection unit 113 selects image data G11 to G13. When selecting image data G in real time, the selection unit 113 determines the distance based on the driving state from the wheel drive mechanism and steering mechanism 106. After driving ends, the selection unit 113 selects one image data based on the driving information associated with the image data stored in the still image storage unit 112.

Figure 5 is a diagram illustrating the acquisition of a recognition window for a crop S and the use of the same for synthesis. As shown in the figure, recognition processing is performed on the crop S contained in each of the image data G11 to G13. As shown in Figure 5(b), recognition window N11 and other recognition windows N are acquired from image data G11. These recognition windows are detection windows in object detection. Similarly, recognition windows N12, N12a, and other recognition windows N are acquired from image data G12, and recognition window N13 and other recognition windows N are acquired from image data G13.

Then, the image processing unit 115 aligns the recognition window N11 with the recognition window N12. Based on the driving information of the selected image data and the position of the recognition window N1 or N12 in the image data, it is determined whether the recognition window N1 and the recognition window N12 are crops S in the same position, i.e., whether they are the same crop S.

Similar alignment is performed for the other recognition windows N. This process links image data G11 and image data G12. Image data G12 and image data G13 are also linked by the same process.

Figure 5 (c) shows the composite image data X obtained by concatenating the data. For ease of explanation, image data G11 to G13 have been used in the explanation, but the above processing is performed on all image data G that were photographed and selected in the farm field H.

Note that the crops S and other items are not necessarily arranged to be easily connected according to the shape of the image data G. Depending on the driving state of the automated driving device 100, the crops S may be arranged diagonally with respect to the shape of the image data (rectangle), as shown in image data G1 and G2 in FIG. 3.

Even in such a case, as shown in FIG. 6, the orientation of the image data G21 to G23 is changed and linked to match the recognition window N. In this case, the image data may be distorted, but since the purpose of this disclosure is not to obtain a detailed image of the crop S but to determine the condition of the crop S, the distortion is not a problem.

Next, the operation of the automatic driving device 100 in this disclosure will be described. FIG. 7 is a flowchart showing the operation of the automatic driving device 100. In this operation, it is assumed that the automatic driving device 100 is in automatic driving. During automatic driving, an image including one or more crops S in the field H is captured by the camera 117, and image data is acquired (S101). In addition, the behavior of the automatic driving device 100 is detected by the behavior detection unit 111 (S102). The captured image data, behavior information, and driving information (distance, etc.) are stored in the still image storage unit 112 (S103).

Then, image data is selected by the selection unit 113 (S104). At that time, image data captured when traveling a predetermined distance and its behavior information are selected. At this time, if an abnormal value is indicated in the behavior information, the image data corresponding to that is not adopted, and either image data before or after that image data is selected. Note that if there is an abnormality in the behavior information of either the image data before or after that, it is advisable to select yet another image data.

Figure 8 shows abnormal values of behavior information. As shown in the figure, the behavior information shows the time axis on the horizontal line and the behavior numerical value on the vertical line. In Figure 8, yaw, pitch, and roll are shown as the behavior numerical value, but this is not limited to these. If the behavior information (behavior numerical value) of the selected image data is extremely different from other behavior information, for example, if it deviates from the average value by more than a predetermined value, the behavior information is determined to be abnormal.

When there is an abnormality in the behavior information, the image data may contain momentary blurring, making it inappropriate as an image. In order to exclude such an abnormality, it is advisable to select image data with normal behavior information before and after the abnormality. Note that the camera 117 periodically performs image capture processing, and the still image storage unit 112 stores multiple image data containing the same range.

Next, the image processing unit 115 recognizes the crop S in the image data and generates a recognition window (S105). The image processing unit 115 performs a linking process of the image data using the recognition window and generates composite image data (S106). The composite image data is stored in the composite image storage unit 116 (S107).

The above process is performed while driving. Steps S101 to S103 are performed while driving, and steps S104 to S107 may be performed by an administrator or other user after driving has ended.

A modified example of the automatic driving device 100 in the present disclosure will be described. FIG. 9 is a diagram showing the functional configuration of the automatic driving device 100a in the modified example. The automatic driving device 100a includes a camera 101, a recognition unit 102, a perspective transformation unit 103, an adjustment unit 104, a speed control unit 105, a wheel drive mechanism and a steering mechanism 106, a working unit 107, a behavior detection unit 111, a still image storage unit 112a, a synthesis processing unit 112b, a selection unit 113, an image processing unit 115, a synthesis image storage unit 116, and a camera 117.

The main difference between this and the automated driving device 100 is that it is equipped with a synthesis processing unit 112b.

In this modified example, the still image storage unit 112a stores image data of the driving direction (forward) of the automated driving device 100a captured by the camera 101, and image data of the rear (or lower rear) of the automated driving device 100a captured by the camera 117.

Then, the synthesis processing unit 112b synthesizes the image data of the driving direction (forward) of the automatic driving device 100a captured by the camera 101 with the image data of the rear (or below the rear) of the automatic driving device 100a captured by the camera 117. FIG. 10 is an explanatory diagram showing the procedure of the synthesis process. As shown in the figure, the synthesis processing unit 112b cuts out image data G101, which is a portion of the image data of the forward direction captured by the camera 117, and also cuts out image data G102, which is a portion of the image data of the rearward direction captured by the camera 117. The image data G101 and image data G102 are then combined.

The recognition unit 102 performs a recognition process for the crop S on the combined image obtained by the synthesis processing unit 112b. After the recognition process, the recognition unit 102 separates the image data G101 and G102, and passes the image data G101 to the perspective transformation unit 103 and the image data G102 to the selection unit 113, where they are processed.

That is, the image data G101 is used for automatic driving control in the automatic driving device 100, and the image data G102 is used for a composite image for understanding the condition of the field H.

This configuration allows the recognition unit 102 to perform processing all at once, making it possible to speed up the processing. The upper part of the image data of the front captured by the camera 101 and the upper part of the image data of the publication captured by the camera 117 are not important parts for the adjustment processing by the adjustment unit 104 and for combining the image data. Therefore, the synthesis processing unit 112b reduces the processing load by performing synthesis processing on the parts excluding the unimportant parts and performing recognition processing all at once.

Next, the effects of the automated driving device 100 disclosed herein and the automated driving device 100a in its modified example will be described.

This automated driving device 100 is a device for performing a predetermined task on crops S, which are objects arranged in the direction of travel. As the automated driving device 100 moves, the camera 117, which functions as an image generation photographing unit, photographs multiple crops S arranged in multiple rows in the direction of travel. The still image storage unit 112 stores image data including the photographed multiple crops S. The selection unit 113 selects multiple image data photographed as the automated driving device 100 moves (e.g., after a predetermined distance has been traveled or after a predetermined time has elapsed). The image processing unit 115 links the selected multiple image data.

Here, the camera 117, which is the image generating imaging unit, captures multiple images while shifting the specified range as the vehicle moves so as to include all of the crops S that are included in the specified range at the time of capture.

When selecting multiple images, the selection unit 113 selects an image (image G1 in FIG. 4) of a first predetermined range (field H11 in FIG. 4), and selects an image (image G12) of a second predetermined range (field H12) that includes a portion of the crop S in the predetermined range (field H11).

At that time, the selection unit 113 selects an image of the second predetermined range that is defined to include objects arranged behind the first predetermined range in the direction of travel on the direction of travel side of the second predetermined range.

With this configuration, the automatic driving device photographs and connects the crops S in the field H while performing a specified task by automatic driving, thereby obtaining image data of the entire field H. The manager can determine the condition of the crops S based on that image data. This allows the manager to efficiently plan yields or deal with pests and diseases in the field H.

In this automated driving device 100, the image processing unit 115 recognizes the multiple crops S photographed and acquires a recognition window that is the recognition area. Then, the multiple images are linked based on the recognition window included in the multiple image data.

This configuration allows multiple image data to be linked more accurately. For example, accurate linking is possible by aligning the recognition windows of multiple image data.

The autonomous driving device 100 further includes a behavior detection unit 111 that detects its behavior. The behavior indicates yaw, pitch, roll, acceleration, etc. When storing image data, the still image storage unit 112 stores the image data in association with behavior information indicating the behavior. When the behavior information of the selected image data indicates a predetermined situation, the selection unit 113 replaces the selected image with at least one image before or after the selected image.

With this configuration, image data that is determined to be abnormal based on behavior information can be replaced with other image data. Therefore, appropriate image data can be generated without linking image data that is inappropriate for judging the condition of the field, such as blurred image data.

The automated driving device 100a in the modified example is a device that performs a predetermined task on objects arranged in the traveling direction. In this automated driving device 100a, the camera 101, which is a control imaging unit, captures an image of the crop S in the traveling direction of the automated driving device 100a to obtain image data. The adjustment unit 104 controls the traveling direction based on the recognition result of the object contained in the image data captured by the camera 101. The recognition unit 102 recognizes the crop S in image data that is a combination of image data captured by the camera 117, which functions as an image generation imaging unit, and image data captured by the camera 101.

This configuration allows the recognition unit 102 to perform recognition processing collectively, thereby reducing the processing load, etc.

The automated driving device 100, which performs image capture while traveling, may also include a camera 117, which is an image capture unit for generating images, which captures images of a predetermined range while traveling to include multiple objects arranged in multiple rows in the direction of travel, and an image processing unit 115 that recognizes multiple objects in the predetermined range and combines multiple images (schematic data) based on the recognition results. The combined image is stored in a composite image storage unit 116.

That is, in this automated driving device 100, the image processing unit 115 may recognize the crop S from the image data of the field H, generate schematic data (recognition result data) consisting only of the recognition window N, and link the image data based on the recognition window N. For example, this is data such as that shown in FIG. 6 (data G21 to G23). Although FIG. 6 uses a schematic diagram for ease of explanation, the image processing unit 115 may generate and link such schematic data (recognition result data).

The size of the recognition window N changes depending on the growth of the crop S, and it is possible to grasp the growth of the crop S from the size of the recognition window N. Therefore, by linking data consisting of only the recognition window N to obtain data for the entire field H, it is possible to easily map the analysis results. The size of the recognition window N changes depending on the size of the crop, and as a result, the recognition results alone may be sufficient to grasp the distribution of crop size (to know areas where growth is poor).

Note that the automatic driving device 100 in this disclosure and the automatic driving device 100a in the modified example are not limited to devices that perform predetermined tasks on objects arranged in the direction of travel, or to automatic driving devices that travel automatically. For example, the driving itself may be operated by a person via radio control, and a radio-controlled device that automates the photographing task may be used. Also, a tractor with a driver on board may be equipped with this photographing function and image processing function. Automatic photographing can be performed while performing tasks such as spraying pesticides or top dressing. In either form, it is sufficient that the device functions as a photographing device that photographs the location it has traveled or moved to.

In the above disclosure, the autonomous driving device 100 performs image processing, but this is not limited to this. Image data and behavior information obtained by the autonomous driving device 100 may be acquired by an acquisition unit of an information processing device (computer, etc.) not shown, and linking processing may be performed on the image data. The information processing device has the functions of a still image storage unit 112, a selection unit 113, an image processing unit 115, and a composite image storage unit 116.

The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or may be realized using two or more devices that are physically or logically separated and directly or indirectly connected (e.g., using wires, wirelessly, etc.). The functional blocks may be realized by combining the one device or the multiple devices with software.

Functions include, but are not limited to, judgement, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, regard, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs the transmission function is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on the method of realization for either of these.

For example, the autonomous driving device 100 according to an embodiment of the present disclosure may function as a computer that performs processing of the autonomous driving method of the present disclosure. FIG. 11 is a diagram showing an example of a hardware configuration of a control part of the autonomous driving device 100 according to an embodiment of the present disclosure. The autonomous driving device 100 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

In the following description, the term "apparatus" may be interpreted as a circuit, device, unit, etc. The hardware configuration of the automated driving device 100 may be configured to include one or more of the devices shown in the figure, or may be configured to exclude some of the devices.

Each function of the automated driving device 100 is realized by loading a specific software (program) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communications via the communication device 1004, and control at least one of reading and writing data in the memory 1002 and storage 1003.

The processor 1001, for example, operates an operating system to control the entire computer. The processor 1001 may be configured with a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. For example, the above-mentioned recognition unit 102, perspective transformation unit 103, adjustment unit 104, speed control unit 105, selection unit 113, image processing unit 115, etc. may be realized by the processor 1001.

The processor 1001 also reads out programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. The programs used are those that cause a computer to execute at least a part of the operations described in the above-mentioned embodiments. For example, the recognition unit 102 of the automatic driving device 100 may be realized by a control program stored in the memory 1002 and running on the processor 1001, and other functional blocks may be similarly realized. Although the above-mentioned various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may be transmitted from a network via a telecommunications line.

The memory 1002 is a computer-readable recording medium, and may be composed of at least one of, for example, a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing an autonomous driving method according to one embodiment of the present disclosure.

Storage 1003 is a computer-readable recording medium, and may be, for example, at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, and the like. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other suitable medium including at least one of memory 1002 and storage 1003.

The communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc., to realize, for example, at least one of Frequency Division Duplex (FDD) and Time Division Duplex (TDD).

The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. Note that the input device 1005 and the output device 1006 may be integrated into one configuration (e.g., a touch panel).

In addition, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

The automated driving device 100 may also be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

The processing steps, sequences, flow charts, etc. of each aspect/embodiment described in this disclosure may be reordered unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order and are not limited to the particular order presented.

Each aspect/embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation.

Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended as an illustrative example and does not have any limiting meaning on the present disclosure.

Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

Software, instructions, information, etc. may also be transmitted and received via a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and/or wireless technologies (such as infrared, microwave), then these wired and/or wireless technologies are included within the definition of a transmission medium.

The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

Note that the terms described in this disclosure and the terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of the channel and the symbol may be a signal (signaling). Also, the signal may be a message. Also, a component carrier (CC) may be called a carrier frequency, a cell, a frequency carrier, etc.

As used in this disclosure, the terms "system" and "network" are used interchangeably.

In addition, the information, parameters, etc. described in this disclosure may be represented using absolute values, may be represented using relative values from a predetermined value, or may be represented using other corresponding information. For example, radio resources may be indicated by an index.

The names used for the parameters described above are not intended to be limiting in any way. Furthermore, the formulas etc. using these parameters may differ from those explicitly disclosed in this disclosure. The various channels (e.g., PUCCH, PDCCH, etc.) and information elements may be identified by any suitable names, and therefore the various names assigned to these various channels and information elements are not intended to be limiting in any way.

As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Additionally, "judgment" and "decision" can include considering resolving, selecting, choosing, establishing, comparing, etc., to have been "judged" or "decided." In other words, "judgment" and "decision" can include considering some action to have been "judged" or "decided." Additionally, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

The terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

The "means" in the configuration of each of the above devices may be replaced with "part," "circuit," "device," etc.

When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Additionally, the term "or," as used in this disclosure, is not intended to be an exclusive or.

In this disclosure, where articles have been added through translation, such as a, an, and the in English, this disclosure may include that the nouns following these articles are in the plural form.

In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

100...automatic driving device, 101...camera, 102...recognition unit, 103...perspective conversion unit, 104...adjustment unit, 105...speed control unit, 106...wheel activation mechanism and steering mechanism, 107...working unit, 111...behavior detection unit, 112...still image storage unit, 113...selection unit, 115...image processing unit, 116...synthetic image storage unit, 117...camera, 100a...automatic driving device, 112a...still image storage unit, 112b...synthetic processing unit.

Claims (3)

In a photographing device that performs photographing processing while moving,
an image generation photographing unit that photographs a predetermined range including a plurality of objects arranged in a plurality of rows in a moving direction of the photographing device while the photographing device is moving;
A storage unit that stores the captured image of the predetermined range;
a selection unit that selects a plurality of images captured in accordance with the movement of the photographing device;
an image processing unit that connects the selected images ;
The image processing unit includes:
An imaging device that recognizes a plurality of objects in the images to obtain recognition areas, and connects the plurality of images by aligning the recognition areas in each image based on the recognition areas contained in the plurality of images . In a photographing device that performs photographing processing while moving,
an image generation photographing unit that photographs a predetermined range including a plurality of objects arranged in a plurality of rows in a moving direction of the photographing device while the photographing device is moving;
A storage unit that stores the captured image of the predetermined range;
a selection unit that selects a plurality of images captured in accordance with the movement of the photographing device;
an image processing unit that connects the selected images;
a behavior detection unit that detects a behavior of the photographing device ,
the image generation photographing unit photographs a plurality of times while shifting the predetermined range according to travel so as to include all of the crops of the target included in the predetermined range at the time of photographing,
The storage unit stores the image of the predetermined range in association with the behavior,
When the behavior corresponding to the selected image indicates a predetermined situation, the selection unit replaces the selected image with an image at least one chronologically preceding or succeeding the selected image.
Filming equipment. In a photographing device that performs photographing processing while moving,
an image generation photographing unit that photographs a predetermined range including a plurality of objects arranged in a plurality of rows in a moving direction of the photographing device while the photographing device is moving;
A storage unit that stores the captured image of the predetermined range;
a selection unit that selects a plurality of images captured in accordance with the movement of the photographing device;
an image processing unit that connects the selected images;
a control image capturing unit for capturing an image of an object in a traveling direction of the image capturing device;
A control unit that controls a traveling direction based on a recognition result of an object included in the driving control image,
The image processing unit recognizes the object on a combined image obtained by combining the image captured by the image generation photographing unit and the driving control image captured by the control photographing unit.
Filming equipment.

Images