JP7660264B2 - OBJECT DETECTION DEVICE, OBJECT DETECTION METHOD, AND OBJECT DETECTION SYSTEM - Google Patents

Description

The present disclosure generally relates to object detection devices, object detection methods, and object detection systems.

In recent years, with the continued growth of information technology and the Internet of Things (IoT), sensor devices are increasingly being used to collect data in a wide variety of fields. This collected data can then be analyzed to detect and measure various physical phenomena and glean valuable insights.

One aspect of information technology and IoT systems relates to object detection and tracking in images collected by cameras. Object detection and tracking has applications in the surveillance, health care, retail, and transportation industries. Objects of interest may include, for example, humans, animals, vehicles, luggage, etc. However, in some cases, an object of interest in an image may be only partially visible due to occlusion by other objects. In such cases, existing object detection methods have difficulty providing accurate object detection results.

Therefore, in view of such challenges, methods have been proposed for detecting occluded objects in an image.
As an example, U.S. Patent No. 5,399,633 discloses a method, which "includes the following steps: a video image sequence including detection results from one or more detectors is received, the detection results identifying one or more objects. A clustering framework is applied to the detection results to identify one or more clusters associated with the one or more objects. The clustering framework is applied to the video image sequence on a frame-by-frame basis. Spatial and temporal information for each of the one or more clusters is determined. The one or more clusters are associated with the detection results based on the spatial and temporal information in successive frames of the video image sequence to generate tracking information. One or more target tracks are generated based on the tracking information for the one or more clusters. The one or more target tracks are integrated to generate refined tracks for the one or more objects."

In addition, Patent Document 2 discloses the following: "An object detection device and method using a fusion sensor are disclosed. The method for detecting an object using a fusion sensor includes: obtaining a search area for detecting an object, collecting lidar data and radar data using a lidar sensor and a radar sensor; extracting an object present in the search area based on the collected lidar data and radar data, respectively; setting a region of interest for the lidar sensor using the collected lidar data; generating an occlusion depth using the object extracted based on the lidar data; setting a region of interest for the radar sensor using the collected radar data; and determining whether an object present in the search area is a moving object by using the region of interest or occlusion depth of the lidar sensor and the region of interest of the radar sensor."

U.S. Pat. No. 10,002,309 B2 Korean Patent No. 102090487B1

US Patent No. 5,399,633 discloses a technique in which once a target object is accurately identified in a particular frame, a clustering method is used to track the target object across subsequent frames in which it may be occluded. The clustering method of US Patent No. 5,399,633 requires the collection of a video sequence, and clusters are associated with the target object based on spatial and temporal information of the target object in successive frames.

However, the technique disclosed in U.S. Patent No. 6,399,633 primarily relies on accurate identification of a target object in a frame in which it is unoccluded before it can be tracked in a subsequent frame in which it may be occluded. Thus, the technique disclosed in U.S. Patent No. 6,399,633 is not suitable when a video sequence in which the target object is unoccluded in one or more frames is not available. Furthermore, because the technique disclosed in U.S. Patent No. 6,399,633 performs clustering based on the temporal and spatial relationships of target objects in subsequent frames of the video sequence, clustering cannot be performed accurately in situations with slow image capture rates when the distance of target objects between subsequent frames is too large for cluster association.

Patent document 2 discloses a sensor fusion method for determining the depth of a partially occluded object based on both radar and lidar measurements.

However, the technique disclosed in U.S. Patent No. 5,399,633 focuses on determining the depth of partially occluded objects and has limited ability to detect the class of the target object. Therefore, the technique of U.S. Patent No. 5,399,633 is not suitable for situations where accurate class identification of the target object is important, such as in automotive or rail applications where the danger of a particular object to the safety of the vehicle or train may depend on the class of the object (e.g., dynamic objects may pose a greater risk than static objects).

Therefore, an object of the present disclosure is to provide a device, method, and system for object detection that can detect occluded objects with high accuracy.

One representative example of the present disclosure is an object detection device, comprising an image acquisition unit for receiving a target image including at least a first target object, an image segmentation unit for processing the target image to identify occluded regions of the target image, in which the at least first target object is at least partially occluded by a first occluding object, and to assign a first object label to the first occluding object, the first object label being indicative of an object class of the first occluding object, and an image transformation unit for generating a first recovered image by transforming the occluded regions of the target image into a first recovered mask based on the first object label. The present invention relates to an object detection device comprising: a generator unit for processing the first recovered image to generate a first recovered image in which the first recovered mask is replaced with a first predicted image including at least the first predicted object associated with a second object label indicating an object class corresponding to both the first predicted object and the first target object; and an object detection unit for processing the first recovered image to detect the first target object and generating an object detection result including at least a location of the first target object and a target object label indicating an object class of the first target object.

The present disclosure makes it possible to provide a device, method, and system for object detection that can detect occluded objects with high accuracy.

Problems, configurations, and advantages other than those described above will become clear from the following description of an embodiment for carrying out the invention.

FIG. 1 illustrates an exemplary computing architecture for implementing embodiments of the present disclosure. FIG. 1 illustrates an exemplary configuration of an object detection system, according to an embodiment. FIG. 2 is a block diagram of an inference stage process of an object detection system, according to an embodiment. FIG. 2 is a block diagram of a training phase process of an object detection system, according to an embodiment. FIG. 2 is a block diagram of an image segmentation training process, according to an embodiment. FIG. 2 is a block diagram of a set of transformation units according to an embodiment. 3A-3C are diagrams illustrating examples of a target image, a restored image, and a restored image, according to an embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. It should be noted that the embodiments described herein are not intended to limit the present invention according to the claims, and it should be understood that each of the elements and their combinations described with respect to the embodiments are not strictly necessary to implement aspects of the present invention.

Various aspects are disclosed in the following description and in the associated drawings. Alternative aspects may be devised without departing from the scope of the present disclosure. Additionally, well-known elements of the present disclosure will not be described in detail or will be omitted so as not to obscure the relevant details of the present disclosure.

The words "exemplary" and/or "example" are used herein to mean "serving as an example, instance, or illustrative example." Any aspect described herein as "exemplary" and/or "example" is not necessarily to be construed as preferred or advantageous over other aspects. Similarly, the phrase "aspects of the present disclosure" does not require that all aspects of the disclosure include the discussed feature, advantage, or characteristic of operation.

Furthermore, many aspects are described in terms of sequences of actions to be performed, for example, by elements of a computing device. It will be appreciated that the various actions described herein can be performed by specific circuitry (e.g., an application specific integrated circuit (ASIC)), program instructions executed by one or more processors, or a combination of both. In addition, the sequences of actions described herein can be considered to be embodied as a whole in any form of computer-readable storage medium that stores a corresponding set of computer instructions that, when executed, can cause an associated processor to perform the functions described herein. As such, various aspects of the present disclosure may be embodied in many different forms, all of which are contemplated to be within the subject matter set forth in the claims.

Aspects of the present disclosure relate to using semantic segmentation to identify occluding objects that occlude a target object, replacing regions of the occluding objects with a recovery mask, and then using a generative adversarial neural network (GAN) to generate a recovered image in which the recovery mask is replaced with a predicted image showing at least predicted objects that share an object class with the target object. By performing object detection on this recovered image, high accuracy object detection results can be achieved.

More specifically, to recognize occluded regions in an image, high-resolution semantic segmentation can be utilized to identify occluding objects that occlude target objects. Here, for railroad or automotive applications, examples of occluding objects may include utility poles, trees, and other barriers, and target objects may include people, animals, automobiles, etc. Thus, as used in this disclosure, "occluded region" refers to a region in an image that corresponds to an occluding object identified by semantic segmentation.

During the training phase, the occluded regions identified using high-resolution semantic segmentation can be used to estimate model weights for the patch label classifier, discriminator, and generator in the GAN for their ability to recover occluded objects. Additionally, during the inference phase, the occluded regions can be used to guide the GAN to generate predicted images that are replaced by the occluded regions, where the occluded regions can be represented using a binary recovery mask.

Furthermore, during the training phase, the generator in the GAN may include a global/patch discriminator and additional patch label classifiers to guide the training of the generator to recover the object class of the target object with higher accuracy. More specifically, these global/patch discriminators may receive the occluded regions identified using semantic segmentation to generalize a model for the occluded regions from a set of labels, and guide the generator to generate realistic predicted images that replace the occluded regions.

Further, in embodiments, aspects of the present disclosure relate to utilizing multiple transformation units (e.g., additional GANs) to improve the quality of the input image and modify features (e.g., weather conditions, lighting conditions). The outputs of these transformation units can be weighted and combined into a single image using a set of convolutional layers. By fusing the results of multiple transformation units, it is possible to balance the weights of each transformation unit and reduce the chance of false positives.

Thus, according to the present disclosure, it is possible to provide a device, method, and system for object detection that can detect occluded objects with high accuracy.

Referring now to the drawings, FIG. 1 is a schematic block diagram of a computer system 100 for implementing various embodiments of the present disclosure, according to an embodiment. The mechanisms and apparatus of the various embodiments disclosed herein are equally applicable to any suitable computing system. The major components of the computer system 100 include one or more processors 102, memory 104, terminal interface 112, storage interface 113, I/O (input/output) device interface 114, and network interface 115, all of which are communicatively coupled, directly or indirectly, for inter-component communication via memory bus 106, I/O bus 108, bus interface unit 109, and I/O bus interface unit 110.

Computer system 100 may include one or more general purpose programmable central processing units (CPUs) 102A and 102B, generally referred to herein as processors 102. In embodiments, computer system 100 may include multiple processors, although in certain embodiments computer system 100 may alternatively be a single CPU system. Each processor 102 executes instructions stored in memory 104 and may include one or more levels of on-board cache.

In an embodiment, memory 104 may include random access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing or encoding data and programs. In a particular embodiment, memory 104 represents the entire virtual memory of computer system 100 and may further include virtual memory of other computer systems coupled to computer system 100 or connected via a network. While memory 104 can be conceptually viewed as a single monolithic entity, in other embodiments memory 104 is a more complex configuration, such as a hierarchy of caches and other memory elements. For example, memory may exist in multiple levels of caches, which may be further divided by function, whereby one cache holds instructions and another cache holds non-instruction data used by the processor. Memory may also be distributed and associated with different CPUs or sets of CPUs, as known in any of a variety of so-called non-uniform memory access (NUMA) computer architectures.

Memory 104 may store all or a portion of various programs, modules, and data structures for handling data transfers as described herein. For example, memory 104 may store object detection application 150. In an embodiment, object detection application 150 may include instructions or statements that are executed on processor 102 or interpreted by processor 102 to perform functions as described further below.
In certain embodiments, object detection application 150 is implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and/or other physical hardware devices in place of or in addition to a processor-based system. In embodiments, object detection application 150 may include data in addition to instructions or statements. In certain embodiments, a camera, sensor, or other data input device (not shown) may be provided in direct communication with bus interface unit 109, processor 102, or other hardware of computer system 100. In such a configuration, the need for processor 102 to access memory 104 and object detection application 150 may be reduced.

Computer system 100 may include a bus interface unit 109 that handles communication between processor 102, memory 104, display system 124, and I/O bus interface unit 110. I/O bus interface unit 110 may be coupled to I/O bus 108 to transfer data to and from various I/O units. I/O bus interface unit 110 communicates via I/O bus 108 with a number of I/O interface units 112, 113, 114, and 115, also known as I/O processors (IOPs) or I/O adapters (IOAs). Display system 124 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both types of data to display device 126. Additionally, computer system 100 may include one or more sensors or other devices configured to collect and provide data to processor 102.
By way of example, computer system 100 may include biometric sensors (e.g., collecting heart rate data, stress level data), environmental sensors (e.g., collecting humidity data, temperature data, pressure data), motion sensors (e.g., collecting acceleration data, movement data), etc. Other types of sensors are possible. Display memory may be dedicated memory for buffering video data. Display system 124 may be coupled with a display device 126, such as a standalone display screen, a computer monitor, a television, a tablet, or the display of a handheld device.
In one embodiment, display device 126 may include one or more speakers for rendering audio. Alternatively, one or more speakers for rendering audio may be coupled with the I/O interface unit. In an alternative embodiment, one or more of the functions provided by display system 124 may be incorporated in an integrated circuit that also includes processor 102. Additionally, one or more of the functions provided by bus interface unit 109 may be incorporated in an integrated circuit that also includes processor 102.

The I/O interface unit supports communication with various storage and I/O devices. For example, the terminal interface unit 112 supports the connection of one or more user I/O devices 116, which may include user output devices (such as a video display device, speakers, and/or a television set) and user input devices (such as a keyboard, mouse, keypad, touchpad, trackball, buttons, light pen, or other pointing device). A user may use a user interface to manipulate the user input devices to provide input data and commands to the user I/O devices 116 and the computer system 100, and may also receive output data via the user output devices. For example, the user interface may be presented via the user I/O devices 116, such as displayed on a display device, played through speakers, or printed by a printer.

The storage interface 113 supports the connection of one or more disk drives or direct access storage devices 117 (typically rotating magnetic disk drive storage devices, but may alternatively be other storage devices, including arrays of disk drives or solid state drives such as flash memory that appear to the host computer as a single mass storage device). In some embodiments, the storage device 117 may be implemented by any type of secondary storage device. The contents of the memory 104, or any portion thereof, may be stored in the storage device 117 and retrieved from the storage device 117 as needed. The I/O device interface 114 provides an interface to any of a variety of other I/O devices, or other types of devices, such as printers or fax machines. The network interface 115 provides one or more communication paths from the computer system 100 to other digital devices and computer systems, which may include, for example, one or more networks 130.

1 illustrates a particular bus structure providing direct communication paths between the processor 102, memory 104, bus interface 109, display system 124, and I/O bus interface unit 110, in alternative embodiments, the computer system 100 may include different buses or communication paths that may be configured in any of a variety of forms, such as point-to-point links in a hierarchical, star, or web configuration, multiple hierarchical buses, parallel and redundant paths, or any other suitable type of configuration. Additionally, while the I/O bus interface unit 110 and the I/O bus 108 are shown as separate respective components, the computer system 100 may actually include multiple I/O bus interface units 110 and/or multiple I/O buses 108. Although multiple I/O interface units are shown isolating the I/O bus 108 from the various communication paths running to the various I/O devices, in other embodiments, some or all of the I/O devices are directly connected to one or more system I/O buses.

In various embodiments, computer system 100 is a multi-user mainframe computer system, a single-user system, or a server computer or similar device with little or no direct user interface, but which receives requests from other computer systems (clients). In other embodiments, computer system 100 may be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable type of electronic device.

Next, an exemplary configuration of an object detection system according to an embodiment of the present disclosure will be described with reference to FIG. 2.

2 illustrates an exemplary configuration of an object detection system 200 according to an embodiment of the present disclosure. As illustrated in FIG. 2, the object detection system 200 mainly includes an image capture device 210, a client device 220, and an object detection device 230. The image capture device 210, the client device 220, and the object detection device 230 may be communicatively connected via a communication network 225, such as a local area network (LAN) or the Internet.

The image capture device 210 is a device configured to capture image data. The image data captured by the image capture device may include still images or video (i.e., a chronological sequence of image frames). By way of example, the image capture device 210 may include a still camera mounted at a predetermined location (e.g., a surveillance camera), a camera mounted on an automobile or train, a camera included in a mobile computing device (e.g., a smartphone or tablet camera), and the like. In an embodiment, the image capture device may be configured to capture and transmit a target image 212 including a target object to the object detection device 230 via the communication network 225.

The client device 220 may include a computing device for managing the object detection process performed by the object detection device 230 and for viewing and confirming the object detection results 222 sent from the object detection device 230 as a result of the object detection process. In an embodiment, the client device 220 may be used to specify target objects to be detected, set parameters of the object detection device 230, or the like. By way of example, the client device 220 may include a smartphone, a tablet device, a laptop computer, a desktop computer, or other suitable computing device.

The object detection device 230 is a computing device configured to execute an object detection process according to an embodiment of the present disclosure to detect a target object at least partially occluded by an occluded object. As shown in FIG. 2, the object detection device 230 may include an image acquisition unit 232, an image segmentation unit 234, an image transformation unit 236, a generator unit 238, and an object detection unit 240. In an embodiment, the object detection device 230 may be implemented using the computer system 100 shown in FIG. 1, such that the functions of the image acquisition unit 232, the image segmentation unit 234, the image transformation unit 236, the generator unit 238, the object detection unit 240, and other functions of the object detection device 230 are performed using an object detection application 150.
It should be noted that, although an example of an object detection device 230 including an image acquisition unit 232, an image segmentation unit 234, an image transformation unit 236, a generator unit 238, and an object detection unit 240 is shown in FIG. 2, the present disclosure is not limited thereto, and other functional units of the object detection device 230 (e.g., functional units used in the training process) are described herein.

The image acquisition unit 232 is a functional unit configured to receive a target image 212 captured by the image capture device 210. Here, the target image 212 may include an image in which a target object is at least partially occluded by an occluding object. An example in which the target image 212 is a three-channel RGB image is described herein, but the present disclosure is not limited thereto. As an example, if the image capture device 210 is a camera mounted on a driverless train, the image capture device 210 may capture a target image 212 of the surroundings of the driverless train, including a target object for which class identification is desired. Here, an occluding object refers to an object that partially occludes, hides, or obscures a target object in the target image 212. In an embodiment, it is preferable that an occluding object occludes 30% or less of the target object in the target image 212. Referring to the driverless train example, the occluding object may include a railroad crossing device, a utility pole, a tree, a building, or any other object that partially obscures a target object.

The image segmentation unit 234 is a functional unit configured to process the target image acquired by the image acquisition unit 232 to identify occluded regions of the target image 212 where the target object is at least partially occluded by an occluding object and to assign an object label to the occluding object indicative of the object class of the occluding object. Here, the object class refers to a type classification of the object (e.g., person, dog, tree) and the object label refers to a metadata tag indicative of the object class. In an embodiment, the functionality of the image segmentation unit 234 may be realized using a suitable semantic segmentation technique. By way of example, the image segmentation unit 234 may include a fully convolutional network, DeepLab, Atrous convolution, spatial pyramid pooling, global convolutional network, spatio-temporal fully convolutional network, or the like.

The image transformation unit 236 is a functional unit configured to generate a recovered image by transforming occluded regions of the target image 212 into a recovered mask based on the object labels of the occluding objects.

The generator unit 238 is a functional unit for processing the recovered image generated by the image transformation unit 236 to generate a recovered image in which the recovered mask is replaced with the predicted image, where the predicted image is approximately co-located and occupies approximately the same area (e.g., the same area as the occluded region) as the recovered mask. In an embodiment, the generator unit 238 may include a trained generator of a generative adversarial network (GAN), as described below.

The object detection unit 240 is a functional unit configured to process the recovered images generated by the generator unit 238 to detect target objects and generate the object detection results 222. The functionality of the object detection unit 240 may be realized using any suitable object detection technique. By way of example, the object detection unit 240 may include Histogram of Oriented Gradients (HOG), Region-based Convolutional Neural Network (R-CNN), Fast R-CNN, Faster R-CNN, Region-based Fully Convolutional Network (R-FCN), Single Shot Detector (SSD), Spatial Pyramid Pooling (SPP-net), YOLO (You Only Look Once), or the like.

Next, a block diagram of the inference stage process of the object detection system will be described with reference to Figure 3.

In an embodiment, aspects of the present disclosure relate to a training stage process 400 in which the object detection device 230 is trained to achieve high object detection accuracy, and an inference stage process 300 in which the trained object detection device 230 generates object detection results for a target image. Accordingly, FIG. 3 illustrates a block diagram of the inference stage process of the object detection system 200, according to an embodiment.

Initially, in step S310, the image segmentation unit 234 receives the target image 212 captured by the image acquisition unit 232 (not shown in FIG. 3), processes the target image 212 to identify occluded regions of the target image 212 where the target object is at least partially occluded by an occluding object, and assigns an object label to the occluding object. Here, the occluded region of the target image 212 refers to the entire region of the occluding object. The object label refers to a data label that identifies a predicted class of the occluding object. In an embodiment, the object label may also specify the location of the occluding object in the image. As an example, if the target image includes a person as a target object that is partially occluded by a utility pole, in step S310, the image segmentation unit 234 may identify the entire region of the utility pole as the occluded region and assign a first object label of "utility pole" to the utility pole.

Next, in step S320, image transformation unit 236 generates recovered image 325 by transforming the occluded regions of target image 212 into a recovery mask based on the object labels assigned in step S310. Here, recovery mask refers to an image used to indicate and isolate regions of target image 212 to be processed by generator unit 238 in step S330. In an embodiment, the recovery mask may include a binary image in which pixels corresponding to occluded regions are indicated by a first pixel value (e.g., "1") and pixels not corresponding to occluded regions are indicated by a second pixel value (e.g., "0").

Next, in step S330, the generator unit 238 processes the restored image 325 generated by the image transformation unit 236 in step S320 to generate a restored image 335 in which the restored mask is replaced with the predicted image. Here, the predicted image refers to the artificially generated image (i.e., the occluded region) to be replaced with the restored mask. The predicted image includes predicted objects that share an object class with the target object. If the region occluded by the occluding object in the target image 212 includes multiple occluded objects with different object classes, the predicted image includes multiple predicted objects with object classes that respectively correspond to the occluded objects.
As an example, if the target image 212 includes an image in which an occluding object is occluding a person, a car, the ground, and the sky, the generator unit 238 may generate a predicted image in which the predicted objects of the person, the car, the ground, and the sky are replaced with a recovered mask (i.e., the occluded regions).

Next, in step S340, the object detection unit 240 processes the recovered image 335 generated by the generator unit 238 in step S330 to detect the target object and generate an object detection result 222. Here, the object detection result 222 may include a collection of data indicative of at least a target object label indicative of an object class of the target object. In an embodiment, the object detection result 222 may include additional information about the target object, such as the location of the target object in the image, a trajectory (e.g., a predicted movement path) of the target object, or the like. The object detection result 222 may then be transmitted to the client device 220 via the communication network 225.

The inference stage process 300 of the object detection system 200 shown in FIG. 3 can provide a method for detecting occluded objects with high accuracy.

Next, a block diagram of the training phase process of the object detection system is described with reference to Figure 4.

In an embodiment, aspects of the present disclosure relate to a training phase process 400 in which the object detection device 230 is trained to achieve high object detection accuracy, and an inference phase process 300 in which the trained object detection device 230 generates object detection results for a target image. Accordingly, FIG. 4 illustrates a block diagram of a training phase process 400 of an object detection system, according to an embodiment.

First, in step S405, the image segmentation unit 234 is used to process the training image 401 to identify occluding objects (e.g., objects that partially occlude a target object) in the training image 401. The image segmentation unit 234 may output a set of object labels 407 indicating object classes and location information (e.g., coordinates indicating occluded regions corresponding to the occluding objects) of the occluding objects in the training image 401. Here, the training image 401 may include images of a scene in which a target object is occluded by an occluding object. In addition, in step S405, it is assumed that the image segmentation unit 234 has already been trained to accurately identify occluding objects.

In addition, the image segmentation unit 234 may generate a set of pixel classification data 414 by performing a function of labeling each pixel of the training images as “real” or “fake”, such that each pixel that corresponds to an occluded region is labeled as “fake” and each pixel that does not correspond to an occluded region is labeled as “real”. This set of pixel classification data 414 may be used by a discriminator unit, described below, to evaluate the restored images 413 generated by the generator unit 238.

Further, image segmentation unit 234 may use the set of occluded object labels 418 to replace object labels corresponding to occluding objects with object labels corresponding to occluded objects to generate occluded object-labeled images in which the occluded objects are labeled with their correct object classes. More specifically, image segmentation unit 234 may generate occluded object-labeled images by replacing the object label of each pixel in the occluded region with the object label of the occluded object. In an embodiment, the set of occluded object labels 418 may be generated based on a set of annotated target object labels created by a user or administrator (e.g., and further stored in a database) that indicate the correct object classes of the occluded objects in training images 401.

Next, in step S410, the image transformation unit 236 generates a recovered image 411 by transforming the occluded regions of the training image 401 into a recovery mask based on the object label 407. As described herein, the recovery mask may include a binary image in which pixels corresponding to the occluded regions are indicated by a first pixel value (e.g., "1") and pixels not corresponding to the occluded regions are indicated by a second pixel value (e.g., "0").

Next, in step S412, the generator unit 238 processes the recovered image 411 to generate a recovered image 413 (e.g., a second recovered image) in which the recovered mask is replaced with a predicted image showing one or more predicted objects that share an object class with the respective occluded object in the training image 401. Note that at this stage, the accuracy of the recovered image 413 generated by the generator unit 238 in step S412 may be low because the generator unit 238 is not fully trained. However, by repeating the training phase process 400 described herein, the generator unit 238 can be trained to increase its accuracy and generate accurate recovered images 413 in which occluded regions occluded by occluding objects may be replaced with a predicted image including one or more predicted objects having the same object class as the occluded object in the training image 401.

Next, in step S414, the discriminator unit receives the recovered image 413 from the generator unit 238, the pixel classification data 414 from the image segmentation unit 234, and the reference image 415 showing the target object in a state where it is not occluded by the occluding object, and is trained to use the received data to identify regions in the received recovered image 413 that correspond to predicted images. In other words, the discriminator unit is configured to distinguish whether the recovered image 413 is real (e.g., the original image) or fake (e.g., the image generated by the generator unit). Here, the discriminator unit may include a classifier unit of a GAN. In an embodiment, the discriminator unit may include a global discriminator configured to globally evaluate the recovered image 413 and a patch discriminator configured to evaluate predicted images in the recovered image 413.

Here, the reference image 415 may be an image showing substantially the same scene as the training image 401, except that the target object is not occluded by an occluding object. By providing the reference image 415 to the discriminator unit, the discriminator unit may be trained to avoid mislabeling pixels corresponding to the target object. In step S416, using the pixel classification data 414 and the reference image 415, the discriminator unit may be trained to generate a first feedback weight set 417 based on the image quality level of the restored image 413. Here, the image quality level refers to a quantitative measure of the degree to which the restored image 413 is similar to a real image (e.g., a restored image 413 that is more difficult for the discriminator to distinguish as fake may be considered to have a higher image quality level). This first feedback weight set 417 may be back-propagated to the generator unit 238. The parameters of the generator unit 238 may then be adjusted based on this first feedback weight set 417 to facilitate the generation of a more accurate restored image 413.

Next, in step S419, the patch label classifier unit receives the recovered image 413 from the generator unit 238, the set of occluded object labels 418 (e.g., occluded object-labeled images) from the image segmentation unit 234, and the reference image 415, and is trained to classify the object labels of the predicted objects in the received image 413 using the received data. In an embodiment, the patch label classifier unit may compare the object labels identified for the predicted objects in the recovered image 413 with the occluded object labels 418 to evaluate the accuracy of the object labels of the predicted objects in the recovered image 413, and generate a second feedback weight set 420 based on the correlation between the object labels of the predicted objects in the recovered image 413 and the occluded object labels 418. This second feedback weight set 420 may be back-propagated to the generator unit 238. The parameters of the generator unit 238 may then be adjusted based on this second feedback weight set 420 to facilitate the generation of the recovered image 413 with a more accurate predicted object. In other words, the generator unit 238 can be trained to generate recovered images 413 having predicted objects that share the same object class as the occluded target object in the training image 401.

Next, in step S422, the consistency management unit calculates a consistency loss value based on the restored image 413 and the training image 401, which indicates the degree of information loss between the restored image 413 and the training image 401. The consistency loss value is a value that increases with the amount of pixels of the unoccluded regions of the training image 401 that are not present in the restored image 413. The consistency loss value may be back-propagated to the generator unit 238. The parameters of the generator unit 238 may then be adjusted to reduce the consistency loss value for subsequent restored images 413. In this way, the generator unit 238 may be trained to maximize the amount of visual information maintained between the training image 401 and the restored image 413 for unoccluded regions (e.g., to avoid generating a restored image 413 in which unoccluded parts of the training image 401 are missing).

According to the training phase process 400 described above, the discriminator unit can be trained to distinguish whether the recovered images 413 generated by the generator unit 238 are real or fake, and the generator unit 238 can be trained to generate recovered images 413 that fool the discriminator unit (e.g., images that the discriminator unit classifies as real images, despite being generated by the generator unit 238). In this way, by repeating the training phase process 400 until the generator reaches a desired level of accuracy, the generator unit 238 can be trained to generate realistic recovered images in which areas occluded by occluding objects are replaced with predicted images showing objects corresponding to the occluded objects in the original target image.

Next, a block diagram of the training phase process of the image segmentation unit is described with reference to Figure 5.

As described herein, aspects of the disclosure relate to utilizing an image segmentation unit 234 to process a target image acquired by the image acquisition unit 232 to identify occluding objects that at least partially occlude a target object. By way of example, the image segmentation unit 234 may include a fully convolutional network, DeepLab, Atrous convolution, spatial pyramid pooling, a global convolutional network, a spatio-temporal fully convolutional network, or the like. Accordingly, it is desirable to perform an image segmentation training process 500 to train the image segmentation unit 234 to accurately identify occluding objects in a target image. FIG. 5 illustrates a block diagram of an image segmentation training process 500, according to an embodiment.

In step S520, the untrained image segmentation unit 234 is used to process the training images 510. Here, the training images 510 may include images of a scene in which a target object is occluded by an occluding object. The image segmentation unit 234 processes the training images 510 to attempt to identify the object class of the occluding object along with the occluded regions (e.g., areas surrounded by the occluding object). The results output by the image segmentation unit 234 are compared with a set of ground truth data 530 indicating the correct labels and the occluded regions of the occluding objects (e.g., trees, traffic signs, utility poles, railroad crossing devices) in the training images 510, and a loss value is calculated based on the difference between the results of the image segmentation unit 234 and the ground truth data.

Then, parameters of the image segmentation unit 234 are adjusted (e.g., by backpropagation, etc.) to reduce a loss value calculated based on the difference between the results of the image segmentation unit 234 and the ground truth data. By repeating the above process over multiple iterations with different training images 510 and corresponding ground truth data 530, the image segmentation unit 234 may be trained to accurately identify object labels and occluded regions of occluding objects. This image segmentation training process 500 may be repeated until the image segmentation accuracy of the image segmentation unit 234 reaches a predetermined accuracy threshold (e.g., 90%, 95%).
In this manner, the image segmentation unit 234 can be trained to accurately identify object labels and occluded regions of occluding objects.

Next, a block diagram of a set of transformation units is described with reference to Figure 6.

In an embodiment, aspects of the present disclosure relate to the recognition that in some cases, target images may have low image quality due to weather conditions, lighting conditions, image resolution, or the like. In such circumstances, the low image quality may adversely affect the accuracy of object detection performed on these target images. Accordingly, aspects of the present disclosure relate to utilizing a set of transformation units configured to perform image transformation operations on target images to facilitate accurate object detection. Accordingly, FIG. 6 illustrates a set of transformation units 600, in accordance with an embodiment of the present disclosure.

In an embodiment, the set of transformation units 600 may be configured in parallel with the image segmentation unit 234, image transformation unit 236, and generator unit 238 described above. As an example, the set of transformation units 600, image segmentation unit 234, image transformation unit 236, and generator unit 238 may be configured as different layers in a neural network. As described below, the set of output images 640 generated by the set of transformation units 600 and the generator unit 238 may be aggregated into a single composite output.
It should be noted that the image segmentation unit 234, the image transformation unit 236, the generator unit 238, and the restored image 335 have been described above, so their description will be omitted here.

As shown in FIG. 6, the set of transformation units 600 includes a first transformation unit 610, a second transformation unit 620, and a third transformation unit 630. Each of the set of transformation units 600 may be configured using a generator (e.g., a generator of a GAN) trained to perform a different image transformation on the target image 212. More specifically, each of the set of transformation units 600 may be configured to perform an image transformation operation to increase the similarity between the target image 212 and the training images (e.g., object detection training images) used to train the object detection unit 240. By increasing the similarity between the target image 212 and the training images used to train the object detection unit 240, it is possible to facilitate accurate object detection.

The first transformation unit 610 may be configured to perform a resolution transformation operation on the target image 212 to generate a first transformed target image 612 having a different resolution than the target image 212. For example, the first transformation unit 610 may perform a resolution transformation operation on the target image 212 to generate a first transformed target image 612 having an increased resolution with respect to the target image 212 to facilitate object detection.

The second transformation unit 620 may be configured to perform a weather transformation operation to generate a second transformed target image 622 having weather conditions different from the target image 212. For example, if the target image 212 shows a scene having foggy weather, the second transformation unit 620 may perform a weather transformation operation on the target image 212 to generate a second transformed target image 622 in which the target image 212 is transformed into a scene having clear weather to facilitate object detection.

The third transformation unit 630 may be configured to perform an illumination transformation operation to generate a third transformed target image 632 having different illumination conditions than the target image 212. For example, if the target image 212 shows a nighttime scene, the third transformation unit 630 may perform an illumination transformation operation on the target image 212 to generate a third transformed target image 632 in which the target image 212 is transformed into a daytime scene to facilitate object detection.

The transformation management unit 645 may then receive the target image 212, the restored image 335, the first transformed target image 612, the second transformed target image 622, and the third transformed target image 632, and assign a first set of weights to the target images, a second set of weights to the restored image 335, a third set of weights to the first transformed target image 612, a fourth set of weights to the second transformed target image 622, and a fifth set of weights to the third transformed target image 632, where the first, second, third, fourth, and fifth sets of weights are weights indicating the degree to which features corresponding to each respective image should be reflected in the final combined transformed target image.

In an embodiment, the transformation management unit 645 may be trained to assign the first, second, third, fourth, and fifth weight sets based on the degree to which the features of the target image 212, the recovered image 335, the first transformed target image 612, the second transformed target image 622, and the third transformed target image 632 are present in the training images used to train the object detection unit 240, such that images containing features (lighting conditions, weather conditions, resolution) more similar to the training images used to train the object detection unit 240 are given greater weight. In an embodiment, the transformation management unit 645 may be implemented using different model layers of a GAN.

Next, the transformation management unit 645 may combine the target image 212, the restored image 335, the first transformed target image 612, the second transformed target image 622, and the third transformed target image 632 based on the first weight set, the second weight set, the third weight set, the fourth weight set, and the fifth weight set to obtain a combined transformed target image 655. In this way, a combined transformed target image 655 may be generated that reflects the characteristics of each of the target image 212, the restored image 335, the first transformed target image 612, the second transformed target image 622, and the third transformed target image 632 according to the first, second, third, fourth, and fifth weight sets. In other words, a combined transformed target image 655 may be generated that has been transformed to have a greater similarity to the training images used to train the object detection unit 240 .

The combined transformed target image 655 may then be input to the object detection unit 240. The object detection unit 240 may process the combined transformed target image 655 to detect the target object and generate the object detection result 222 as described herein.

The set of transformation units 600 described herein can perform image transformation on the target image 212 to increase the similarity between the target image 212 and the training images used to train the object detection unit 240, thereby facilitating accurate object detection.

Next, examples of the target image, the restored image, and the restored image are described with reference to Figure 7.

As described herein, aspects of the disclosure relate to converting a target image 212, in which one or more target objects are at least partially occluded by one or more occluding objects, into a recovered image 720 in which the regions occluded by the occluding objects are represented as a binary recovered mask, and then generating a recovered image 335 in which the recovered mask is replaced with a predicted image showing predicted objects that correspond to (e.g., share an object class with) the occluded target objects of the target image 212. FIG. 7 shows an example of a target image 212, a recovered image 720, and a recovered image 335, according to an embodiment.

7, a first target object, a car, is occluded by a railway crossing device 705, and a second target object, a person 712, is occluded by a railway crossing device 715. Thus, by processing the target image 212 using the image segmentation unit 234, the region occluded by both railway crossing devices 705 , 715 can be identified and an object label of "railroad crossing device" can be assigned thereto.

The image segmentation unit 234 can then generate the recovered image 720 by transforming the occluded regions of the target image 212 into a recovered mask 725 based on the object labels.

Generator unit 238 can then process recovered image 720 to generate recovered image 335 in which recovered mask 725 is replaced with a first predicted image 745 that includes one or more predicted objects having the same object class as the target object, respectively. More specifically, as shown in recovered image 335, in addition to appropriate background images of sky and ground occluded in target image 212, regions of target image 212 that are occluded by railroad crossing device 705 have been replaced with a predicted image showing car 752, and regions of target image 212 that are occluded by railroad crossing device 715 have been replaced with a predicted image showing person 762.
The recovered image 720 thus generated may then be input to an object detection unit (e.g., object detection unit 240) to facilitate generation of target objects, cars and people, with high accuracy.

According to the object detection device, object detection method, and object detection system described in this disclosure, a generator unit (e.g., a generator in a GAN) can be trained to generate restored images in which occluded regions of a target image are replaced with predicted images showing one or more predicted objects having the same object class as the respective occluded objects in the target image, and then perform object detection on the restored images, thereby generating accurate object detection results even when the target object is occluded.

Additionally, aspects of the present disclosure relate to utilizing multiple transformation units (e.g., additional GANs) to improve the quality of the input image and modify features (e.g., weather conditions, lighting conditions). The outputs of these transformation units can be weighted and combined into a single image using a set of convolutional layers. By fusing the results of multiple transformation units, it is possible to balance the weights of each transformation unit to generate an image that is more similar to the image used to train the object detection unit. Thus, the chance of false positives can be reduced, further improving the accuracy of object detection.

Thus, according to the present disclosure, it is possible to provide a device, method, and system for object detection that can detect occluded objects with high accuracy.

The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to implement aspects of the present invention.

A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read only memories (ROMs), erasable programmable read only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanical encryption devices such as punch cards or groove ridge structures having instructions recorded thereon, and any suitable combination of the above.
As used herein, a computer-readable storage medium should not be construed as being, itself, a primary signal, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted by an electrical wire.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

The computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to manufacture a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus create means for performing the functions/operations specified in the blocks of the flowcharts and/or block diagrams. These computer-readable program instructions may further be stored on a computer-readable storage medium that can cause the computer, programmable data processing apparatus, and/or other device to function in a particular manner, such that the computer-readable storage medium having the instructions stored thereon comprises an article of manufacture that includes instructions that perform aspects of the functions/operations specified in the blocks of the flowcharts and/or block diagrams.

The computer-readable program instructions may further be loaded into a computer, other programmable data processing device, or other device to cause a series of operational steps to be executed on the computer, other programmable device, or other device to create a computer-implemented process, such that the instructions executed on the computer, other programmable device, or other device perform the functions/operations specified in the blocks of the flowcharts and/or block diagrams.

Embodiments according to the present disclosure may be provided to end users via a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing may be defined as a computing facility that provides an abstraction between computing resources and their underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be provisioned and released at high speed with minimal administrative effort or service provider interaction. Cloud computing thereby allows users to access virtual computing resources in the "cloud" (e.g., storage, data, applications, and even comprehensive virtualized computing systems) without regard for the underlying physical systems used to provide the computing resources (or the location of those systems).

The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing a specialized logical function. In some alternative implementations, the functions described in the blocks may occur in a different order than the order described in the drawings. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may be executed in reverse order, depending on the functionality involved. It will also be appreciated that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, may be implemented by a dedicated hardware-based system that executes a specialized function or operation, or executes a combination of dedicated hardware and computer instructions.

Although the above relates to exemplary embodiments of the present invention, other and further embodiments of the present invention may be devised without departing from the basic scope of the present invention, the scope of which is determined by the following claims. The description of various embodiments of the present disclosure has been provided for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein are chosen to explain the principles of the embodiments, practical applications or technical improvements of existing technologies in the market, or to enable others of ordinary skill in the art to which the present disclosure pertains to understand the embodiments disclosed herein.

The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the various embodiments. As used herein, the singular is intended to include the plural as well, unless the context clearly indicates otherwise. "Set", "group", and the like are intended to include one or more. Furthermore, it will be understood that the terms "comprise" and/or "comprising" as used herein specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. In the above detailed description of exemplary embodiments of the various embodiments, reference has been made to the accompanying drawings, in which like numerals indicate like elements, which form a part hereof, by way of example, certain exemplary embodiments are shown and in which the various embodiments may be practiced. The above embodiments have been described in sufficient detail to enable one skilled in the art to practice the embodiments, but other embodiments may be used and logical, mechanical, electrical, and other changes may be made without departing from the scope of the various embodiments. Numerous specific details have been set forth in the above description to provide a thorough understanding of the various embodiments. However, various embodiments may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail so as not to obscure the embodiments.

200 Object detection system 210 Image capture device 212 Target image 220 Client device 222 Object detection result 225 Communication network 230 Object detection device 232 Image acquisition unit 234 Image segmentation unit 236 Image transformation unit 238 Generator unit 240 Object detection unit 300 Inference stage process 335 Recovered image 400 Training stage process 500 Image segmentation training process 600 Set of transformation units

Claims (9)

1. An object detection device, comprising:
an image capture unit for receiving a target image including at least a first target object;
an image segmentation unit for processing the target image to identify occluded regions of the target image, in which at least the first target object is at least partially occluded by a first occluding object, and to assign a first object label to the first occluding object, the first object label being indicative of an object class of the first occluding object;
an image transformation unit for generating a first recovered image by transforming the occluded region of the target image into a first recovered mask based on the first object label;
a generator unit for processing the first recovered image to generate a first recovered image in which the first recovered mask is replaced with a first predicted image including at least the first predicted object associated with a second object label indicating an object class corresponding to both the first predicted object and the first target object;
an object detection unit for processing the first recovered image to detect the first target object and generating an object detection result including a target object label indicating at least a location of the first target object and an object class of the first target object;
An object detection device comprising: the generator unit is further configured to generate, for a training image in which a second target object is at least partially occluded by a second occluding object, a second recovered image in which a second recovered mask corresponding to the second occluding object is replaced with a second predicted image indicative of at least the second predicted object associated with a third object label indicative of an object class of the second predicted object;
the object detection device further comprises a discriminator unit configured to generate a first set of feedback weights based on an image quality level of the second recovered image;
The discriminator unit comprises:
first labeled training images generated by processing the training images with the image segmentation unit to assign a fourth object label to the second occluding object, the fourth object label being indicative of an object class of the second occluding object; and
a reference image including the second target object not occluded by an occluding object;
The object detection device of claim 1 . a patch label classifier unit configured to generate a second set of feedback weights based on a correlation between the third object label of the second predicted object and a fifth object label indicative of an object class of the second target object;
The patch label classifier unit:
second labeled training images generated by processing the training images with the image segmentation unit to assign the fifth object label to the second target object based on a set of annotated target object labels; and
and the reference image.
The object detection device according to claim 2 . The object detection device of claim 3, further comprising a consistency management unit configured to calculate a consistency loss value indicating a degree of information loss between the second recovered image and the training image based on the second recovered image and the training image. The object detection device of claim 4, wherein the generator unit is trained to generate a recovered image based on the first set of feedback weights, the second set of feedback weights, and the consistency loss value. The method further comprises a set of transformation units configured to generate a set of transformed target images, the set of transformation units comprising:
a first transformation unit configured to perform a resolution transformation operation on the target image to generate a first transformed target image having a different resolution than the target image;
a second transformation unit configured to perform a weather transformation operation to generate a second transformed target image having weather conditions different from the target image;
a third transformation unit configured to perform an illumination transformation operation to generate a third transformed target image having a different illumination condition than the target image.
The object detection device of claim 1 . The method further comprises:
assigning a first set of weights to the target image based at least on a similarity between the target image and object detection training images used to train the object detection unit;
assigning a second set of weights to the first recovered image based at least on a similarity between the first recovered image and the object detection training images;
assigning a third set of weights to the first transformed target image based at least on a similarity between the first transformed target image and the object detection training images; and
assigning a fourth set of weights to the second transformed target image based at least on a similarity between the second transformed target image and the object detection training images;
assigning a fifth set of weights to the third transformed target image based at least on a similarity between the third transformed target image and the object detection training images; and
combining the target image, the first restored image, the first transformed target image, the second transformed target image, and the third transformed target image into a combined transformed target image based on the first weight set, the second weight set, the third weight set, the fourth weight set, and the fifth weight set;
The object detection device of claim 6 , configured to: 1. An object detection system, comprising:
an image capture device for capturing a target image including at least a first target object;
an object detection device for detecting the first target object in the target image and generating an object detection result;
a client device for viewing the object detection result;
the image capture device, the object detection device, and the client device are communicatively coupled via a communications network;
The object detection device,
an image acquisition unit for receiving the target image from the image capture device ;
an image segmentation unit for processing the target image to identify occluded regions of the target image, in which at least the first target object is at least partially occluded by a first occluding object, and to assign a first object label to the first occluding object, the first object label being indicative of an object class of the first occluding object;
an image transformation unit for generating a first recovered image by transforming the occluded region of the target image into a first recovered mask based on the first object label;
a generator unit for processing the first recovered image to generate a first recovered image in which the first recovered mask is replaced with a first predicted image including at least a first predicted object associated with a second object label indicating an object class corresponding to both the first predicted object and the first target object;
an object detection unit for processing the first recovered image to detect the first target object, generating the object detection result including a target object label indicating at least a location of the first target object and an object class of the first target object, and transmitting the object detection result to the client device.
Object detection system. 1. A method for object detection, comprising:
generating, using an untrained generator unit, for a training image in which a first target object is at least partially occluded by a first occluding object, a first recovered image in which a first recovered mask corresponding to the first occluding object is replaced with a first predicted image indicative of at least the first predicted object associated with a first object label indicative of an object class of the first predicted object;
training a discriminator unit based on first labeled training images generated by processing the training images with an image segmentation unit and a reference image including the first target object unoccluded by an occluding object, so as to assign a second object label to the first occluding object, the second object label being indicative of an object class of the first occluding object;
generating a first set of feedback weights based on an image quality level of the first restored image by using the discriminator unit to evaluate the first restored image;
training a patch label classifier unit based on the reference images and second labeled training images generated by processing the training images with the image segmentation unit to assign a third object label to the first target object based on a set of annotated target object labels;
using the patch label classifier unit to process the first recovered image to generate a second set of feedback weights based on a correlation between the first object label of the first predicted object and the third object label of the first target object;
Using a consistency management unit, calculate a consistency loss value according to the first recovered image and the training image, the consistency loss value indicating a degree of information loss between the first recovered image and the training image;
generating a trained generator unit by training the untrained generator unit to generate a restored image based on the first set of feedback weights, the second set of feedback weights, and the consistency loss value;
receiving a target image including a second target object using an image capture unit;
processing the target image with the image segmentation unit to identify occluded regions of the target image in which at least the second target object is at least partially occluded by a second occluding object, and assigning a fourth object label to the second occluding object, the fourth object label indicating an object class of the second occluding object;
generating a second recovered image by transforming the occluded region of the target image into a second recovered mask based on the fourth object label using an image transformation unit;
using the trained generator unit for processing the second recovered image to generate a second recovered image in which the second recovered mask is replaced with a second predicted image including at least the second predicted object associated with a fifth object label indicating an object class corresponding to both the second predicted object and the second target object;
and using an object detection unit for processing the second recovered image to generate an object detection result including a target object label indicating at least a location of the second target object and an object class of the second target object.

Images