AU2019360351B2

AU2019360351B2 - Apparatus for validatable output of images

Info

Publication number: AU2019360351B2
Application number: AU2019360351A
Authority: AU
Inventors: Christoph HILKEN; Ervin Kolenovic; Oliver PRENZEL; Peter Schulze
Original assignee: Rheinmetall Electronics GmbH
Current assignee: Rheinmetall Electronics GmbH
Priority date: 2018-10-17
Filing date: 2019-10-14
Publication date: 2023-02-16
Anticipated expiration: 2039-10-14
Also published as: EP3850831A1; DE102018125790A1; AU2019360351A1; CL2021000946A1; WO2020078915A1; BR112021007188A2

Abstract

The invention relates to a device for the validatable output of images, comprising a number of screens for outputting images based on image data and a number of camera devices. Each camera device comprises a camera for providing image data based on a capturing of camera images, and an integration unit for outputting extended image data based on an integration of an encoded information block having a source reference specific to the camera and a time stamp specific to the capture. The device also comprises a transmission channel connected between the screens and the camera devices for transmitting the extended image data from the camera devices to the screens, and a validation unit that can be coupled to the screens for validating the image data that is transmitted via the transmission channel and extended based on the integrated information block.

Description

APPARATUS FOR VALIDATABLE OUTPUT OF IMAGES

The present disclosure relates to an apparatus for validatable output of images of one or more cameras on one or more screens. Furthermore, the present disclosure relates to a vehicle with such an apparatus for validatable output of images of one or more cameras on one or more screens.

For example, at military vehicles such as tanks, apparatuses for outputting im ages are employed which have cameras arranged at the outer skin of the vehicle and at least a screen arranged in the inner room of the vehicle for outputting the images recorded by the cameras. Thereby, a multiplexer is coupled between the cameras and the screen which provides the images of a selected camera from the amount of coupled cameras for the screen.

Thereby, it is possible that a camera arranged at the outer skin of the vehicle supplies a freeze image, that means, it is frozen, and the user within the vehicle has no possibility to notice this.

A particularly high risk arises if the camera is, for example, linked with a weapon station and the image displayed on the screen is used for the shooting on sight.

Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed be fore the priority date of each of the appended claims.

Throughout this specification the word "comprise", or variations such as "com prises" or "comprising", will be understood to imply the inclusion of a stated ele ment, integer or step, or group of elements, integers or steps, but not the exclu sion of any other element, integer or step, or group of elements, integers or steps.

Against this background, the present disclosure provides a solution for validata ble output of images.

Accordingly, an apparatus for validatable output of images is proposed which has a number N1 of screens for outputting images based on image data, with N1 > 1, and a number N2 of camera devices, with N2 > 1. Each one of the camera devices comprises a camera for provisioning image data based on a recording of camera images and an integration unit for outputting extended image data based on an integration of a coded information block which comprises a source indication spe cific for the camera and a time stamp specific for the recording. Further, the ap paratus comprises a transmission channel connected between the N1 screens and the N2 camera devices for transmitting the extended image data from the N2 camera devices to the N1 screens, a validation unit couplable with the N1 screens for validating the extended image data transmitted via the transmission channel on the basis of the integrated information block and a control apparatus which comprises the validation unit and is connected via a respective interface, for ex ample, via Ethernet networking interface, with N1 screen monitoring devices, wherein the control apparatus is configured to initiate or to execute an action in dependence on the validation performed by the validation unit, wherein the con trol apparatus is configured to interlock a weapon couplable with the apparatus if the validation indicates a display of an image of a false camera by the screen.

Advantageously, by the usage of the coded information block comprising the source indication and the time stamp which is imprinted into the image data pre- vious to the transmission channel and is extracted after the transmission chan nel, the entire path through the transmission channel can be validated. Thereby, the extended image data can be, for example, validated based on the information block in such a way whether the correct image source is displayed by the corre sponding screen. Additionally, a concealment of a relevant image, for example, by another video image or a false image composition within the transmission chan nel, a freeze image and the transmission of a cyclic image or also a strongly de layed image can therefore also be detected.

Since the information block is imprinted in the image data provided by the cam era, a secure channel between the recording camera and the representing screen is created advantageously.

The source indication as part of the coded information block and thus as part of specific extended image data is suitable to indicate the camera providing the im age data. Thereby, the source indication can be referred to as identification infor mation or ID or source-ID.

Preferably, the source indication is generated based on a hardware-ID of the camera. The time stamp indicates the time point or the time interval of the recording of the camera images by the camera which provides the image data.

The apparatus can also be referred to as visual system. In particular, a control unit is provided which is configured to control the integration unit as well as the camera. In particular, the control unit is part of the camera device. For example, the control unit comprises an electronic such as a microprocessor or a CPLD (CPLD; Complex Programmable Logic Device).

For example, the camera can be formed as a video camera, as a day view camera, as a thermal imaging camera, as an infrared camera or as a photo camera. In particular, the N2 camera devices can comprise different camera types.

For example, the screen can be formed as a flat screen arranged in a vehicle inte rior of a military vehicle. The screen can also be referred to as monitor.

According to an embodiment, the integration unit is configured to code the coded information block into an area of the extended image data non-visible for an user, in particular into a black-level area of the extended image data.

According to a further embodiment, the integration unit is configured to addition ally code a hash value calculated from the image data into the coded information block.

According to a further embodiment, the integration unit is configured to generate a QR-code such that it has the source indication and the time stamp in coded form.

Advantageously, the usage of a QR-code as information block allows a certain er ror tolerance and thus a configurable permissible coverage rate. According to a further embodiment, the integration unit is configured to code the generated QR-code into an area of the extended image data non-visible for an user, in particular into a black-level area of the extended image data. Thereby, the Least Significant Bit (LSB) is preferably used.

Therefore, the QR-code is preferably coded in the image non-visible for an user und thus can not interfere the user. Preferably, the position of the QR-code is randomly distributed over a central image area, in particular in order to reduce losses regarding the image dynamic.

According to a further embodiment, the integration unit is configured to addition ally code a hash value calculated from the image data into the QR-code.

Advantageously, the calculated and in the QR-code integrated hash values can be used for a downstream image examination, for example, with regard to freeze im ages or with regard to cyclic images. For example, such an image examination can be performed in a downstream screen monitoring device. This is explained in more detail below.

According to a further embodiment, the respective camera device has a camera monitoring device which comprises the integration unit and is connected via a re spective image transmission interface, in particular via Camera Link, with the camera and the transmission channel. The camera monitoring device can also be referred to as camera watchdog.

According to a further embodiment, the validation unit and the respective inte gration unit are time-synchronized by means of a common time basis.

Due to the time-synchronization of the integration unit and the validation unit, it is possible to securely calculate from the transmitted time stamps as part of the information blocks, latency times during the transmission of the image data through the transmission channel.

According to a further embodiment, the respective screen is coupled with the transmission channel via a screen monitoring device. Thereby, the screen moni toring device is connected via a respective image transmission interface, in par ticular via DVI or Camera Link, with the screen and with the transmission chan nel.

According to a further embodiment, the common time basis is correlated with a time of the camera monitoring device and a time of the screen monitoring device.

According to a further embodiment, the common time basis is provided by an ex ternal source.

According to a further embodiment, the screen monitoring device is configured to extract the source indication and the time stamp from the extended image data transmitted via the transmission channel and to transmit the extracted source indication and the extracted time stamp via the interface to the control appa ratus. Thus, the correct assignment between display and recording can be en sured as well as the time delay, that means, the latency of the image data, can be determined.

According to a further embodiment, the screen monitoring device is configured to perform an image examination regarding a freeze image supplied by the camera or repetitive images in the extended image data by means of a comparison of hash values transmitted by the extended image data and/or in dependence on a Fast Fourier transformation.

According to a further embodiment, the transmission channel comprises a com pression unit connected downstream to the camera monitoring devices for com pressing the extended image data, an image distribution unit connected down stream to the compression unit, an image processing unit connected downstream to the image distribution unit and N1 preprocessing units connected downstream to the image processing unit. Thereby, the respective preprocessing unit is as signed to a respective screen monitoring device.

The compression unit can also be referred to as data compression unit. The image distribution unit can also be referred to as video distribution unit. The image pro cessing unit can also be referred to as video processing unit. For example, the im age processing unit can perform a stitching method in order to add together vari ous images of various cameras. The respective preprocessing unit can also be re ferred to as crew terminal processing unit.

For example, if the validation reveals that a specific screen is supposed to output the images of a camera A, but actually outputs the images of a camera B, so the action to be executed can be to interlock a weapon couplable with the apparatus.

According to a further embodiment, the screen monitoring device is configured to fade in an overlay or a black image on the screen if the validation indicates a con cealment of the corresponding camera.

According to a further embodiment, the control apparatus is configured to inter lock the weapon couplable with the apparatus if the validation indicates a con cealment of a relevant image.

For example, a concealment of a relevant image can arise by a false image compo sition in the image distribution unit. Also, in such a case, it is advantageous to interlock the weapon couplable with the apparatus due to safety reasons.

According to a further embodiment, the screen monitoring device is configured to mark the transmitted image data as freeze image if the validation indicates a dis play of a freeze image or a display of a cyclic image or a display of a strongly de layed image. Thereby, a strongly delayed image is an image which has a delay greater than a predetermined threshold.

For example, a cyclic image can arise if a memory is mistakenly read out again and again instead of outputting current video images.

According to a further embodiment, the control apparatus is configured to mark or to suppress a forwarding of the transmitted extended image data if the valida tion indicates a display of a freeze image or a display of a cyclic image or a dis play of a strongly delayed image.

According to a further embodiment, the control apparatus is configured to inter lock a weapon couplable with the apparatus if the validation indicates a display of a freeze image or a display of a cyclic image or a display of a strongly delayed image.

According to a further embodiment, the control apparatus is configured to con trol, in dependence on the validation, a secure failure display which indicates a failure of one or more of the N2 camera devices.

Thus, the user has a secure indication about the fact that a specific camera de vice has failed and preferably which camera has failed. Thereby, the safety of the entire apparatus is significantly increased.

According to a further embodiment, the secure failure display comprises a back ground illumination of the screen, a dedicated display element, for example a LED, arranged at the screen and/or a touch foil arranged at the screen.

The respective unit, for example the validation unit, may be implemented in hardware and/or also in software. When implemented in hardware, the respec tive unit may be implemented as a device or a part of a device, for example as a computer or as a microprocessor or as a FPGA. When implemented in software, the respective unit may be implemented as a computer program product, as a function, as a routine, as part of a programming code or as an executable object.

Furthermore, a vehicle with an apparatus as described above for validatable out put of images is proposed, wherein the cameras of the N2 camera devices are ar ranged at an outer skin of the vehicle and the N1 screens are arranged in a room of the vehicle.

In particular, the vehicle is a military vehicle. Thereby, the vehicle can be a tank, a truck, a special vehicle such as a crane, a loader or an excavator, or a ship.

Further possible implementations also comprise combinations of features or em bodiments described above or below with regard to the embodiments that are not explicitly described. Thereby, the skilled person will also add individual aspects as improvements or additions to the respective basic form of the disclosure.

Examples are described in more detail on the basis of preferred embodiments with reference to the attached figure.

Fig. 1 shows a schematic block diagram of a first embodiment of an appa ratus for validatable output of images.

In Fig. 1, a schematic block diagram of a first embodiment of an apparatus 10 for validatable output of images is shown. The apparatus 10 of Fig. 1 comprises a number N1 of screens 21, 22 for outputting images based on image data BD1, BD2. Without limitation of generality, the apparatus 10 of Fig. 1 has two screens 21, 22 (N1 = 2).

Further, the apparatus 10 comprises a number N2 of camera devices 30, 40, with N2 > 1. Without limitation of generality, the apparatus 10 of Fig. 1 has two cam era devices 30 and 40 (N2 = 2).

Each one of the camera devices 30, 40 comprises a camera 31, 41 for provisioning image data BD1, BD2 based on a recording of camera images and a camera moni toring unit 32, 42. The camera monitoring unit 32, 42 can also be referred to as camera watchdog.

The respective camera 31, 41 is connected to the respective camera monitoring device 32, 42 via an interface, for example, via Camera Link CL.

The respective camera monitoring device 32, 42 comprises an integration unit. For reasons of clarity, the respective integration unit is also provided with the reference numeral 32 or 42.

The integration unit 32, 42 is configured for outputting extended image data eBD1, eBD2 based on an integration of a coded information block. The coded in formation block comprises a source indication Q specific for the camera 31, 41 and a time stamp Z specific for the recording. For example, the extended image data eBD1 which are provided on the output side of the camera device 30, have therefore a source indication Q which indicates the camera 31 and a time stamp Z which indicates the recording by the camera 31 regarding the time point of the recording respectively the time interval of the recording. The same applies to the camera device 40. For example, the integration unit 32, 42 is configured to gener ate a QR-code such that it has the source indication Q and the time stamp Z in coded form. In particular, the generated QR-code is coded into an area of the ex tended image data eBD1, eBD2 non-visible for an user. In particular, the non-vis ible area of the extended image data eBD1, eBD2 is a black-level area of the ex tended image data eBD1, eBD2.

Additionally, the integration unit 32, 42 can be configured to code a hash value calculated from the image data BD1, BD2 into the respective QR-code.

Between the screens 21, 22 and the camera devices 30, 40, a transmission chan nel 50 is connected which is configured for transmitting the extended image data eBD1, eBD2 from the camera devices 30, 40 to the screens 21, 22. Preferably, the transmission channel 50 is based on ethernet. The properties of the transmission channel 50 can vary respectively cannot be known a priori. For this reasons, the transmission channel 50 can also be referred to as black channel. For example, the camera devices 30, 40 are connected to the transmission channel 50 via Cam era Link CL in order to transmit the extended image data eBD1, eBD2 to the transmission channel 50.

The respective screen 21, 22 is coupled via a screen monitoring device 71, 72 with the transmission channel 50. The respective screen monitoring device 71, 72 is connected via a respective digital image transmission interface, for example, via Digital Visual Interface DVI, with the assigned screen 21, 22 and with the trans mission channel 50. Furthermore, to the screen monitoring devices 71, 72, a con trol apparatus 80 is connected, for example, via a dedicated interface, preferably Ethernet networking interface ETH.

The control apparatus 80 comprises a validation unit 60 which is configured to validate the extended image data eBD1, eBD2 transmitted via the transmission channel 50 on the basis of the integrated information block.

Preferably, the validation unit 60 and the respective integration unit 32, 42 are time-synchronized by a common time basis. For example, the common time basis is correlated with a time of the camera monitoring device 32, 42 and a time of the screen monitoring device 71, 72. Additionally or alternatively, the common time basis can be provided by an external source (not shown).

Thereby, the respective screen monitoring device 71, 72 is configured to extract the source indication Q and the time stamp Z from the extended image data eBD1, eBD2 transmitted via the transmission channel 50 and to transmit the ex tracted source indication Q and the extracted time stamp Z via the interface ETH to the control apparatus 80. Based on the extracted source indication Q and the extracted time stamp Z, the validation unit 60 integrated in the control appa ratus 80 can validate the transmitted extended image data eBD1, eBD2. Thereby, the transmitted extended image data eBD1, eBD2 can also be branched off in parallel to the respective screen monitoring device 71, 72, in particular in order to avoid false image outputs by the respective screen monitoring device 71, 72.

In particular, the control apparatus 80 is configured to initiate or to execute an action in dependence on the validation performed by the validation unit 60. For example, the control apparatus 80 can be configured to interlock a weapon cou plable with the apparatus 10 if the validation indicates a display of an image of a false camera 31, 41 by the screen 21, 22.

As the example of Fig. 1 shows, the screen 21 is supposed to display the extended image data eBD1 of the camera device 30 and therefore the camera 31. However, if the validation performed by the validation unit 60 indicates that the extended image data eBD1 of the camera device 30 are not currently being transmitted to the screen 21, but the extended image data eBD2 of the camera 41 of the camera device 40 are being transmitted erroneously, so the control device 80 can inter lock the coupled weapon.

Furthermore, the screen monitoring device 71, 72 can be configured to fade in an overlay or a black image on the respective screen 21, 22 if the validation of the validation unit 60 reveals that the corresponding camera 31, 41 is concealed. Ac cording to a further example, the control apparatus 80 can also interlock the weapon couplable with the apparatus 10 if the validation indicates a concealment of a relevant image. As a further alternative, the screen monitoring device 71, 72 can be configured, in dependence on the validation, to mark the transmitted im age data eBD1, eBD2 as freeze image, for example then if the validation indicates a transmission of a freeze image or a cyclic image or a strongly delayed image.

Additionally, the respective screen monitoring device 71, 72 can be configured to perform an image examination regarding a freeze image supplied by the camera 31, 41 in the extended image data eBD1, eBD2, by means of a comparison of hash values transmitted by the extended image data eBD1, eBD2 and/or in depend ence on a timewise Fast Fourier transformation (FFT).

Although examples of the present invention have been described on the basis of embodiments, the invention can be modified in a variety of ways.

It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the above-described embodiments, without departing from the broad general scope of the present disclosure. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

REFERENCE NUMERALS

10 apparatus 21 screen 22 screen 30 camera device 31 camera 32 integration unit 40 camera device 41 camera 42 integration unit 50 transmission channel 60 validation unit 71 screen monitoring device 72 screen monitoring device 80 control apparatus BD1 image data BD2 image data eBD1 extended image data eBD2 extended image data CL Camera Link DVI Digital Visual Interface ETH Ethernet networking interface

Q source indication Z time stamp

Claims

1. An apparatus for validatable output of images, comprising: a number N1 of screens for outputting images based on image data, with N1 > 1, a number N2 of camera devices, with N2 > 1, wherein each one of the cam era devices includes a camera for provisioning image data based on a recording of camera images and an integration unit for outputting extended image data based on an integration of a coded information block which comprises a source indica tion specific for the camera and a time stamp specific for the recording, wherein the integration unit is configured to additionally code a hash value calcu lated from the image data into the coded information block, a transmission channel connected between the N1 screens and the N2 cam era devices for transmitting the extended image data from the N2 camera devices to the N1 screens, a validation unit couplable with the N1 screens for validating the extended image data transmitted via the transmission channel on the basis of the inte grated information block, and a control apparatus which comprises the validation unit and is connected via a respective interface with N1 screen monitoring devices, wherein the control apparatus is configured to initiate or to execute an action in dependence on the validation performed by the validation unit, wherein the control apparatus is configured to interlock a weapon couplable with the apparatus if the validation indicates a display of an image of a false camera by the screen.

2. The apparatus according to claim 1, wherein the integration unit is configured to code the coded information block into an area of the extended image data non-visible for an user, in particular into a black-level area of the extended image data.

3. The apparatus according to claim 1, wherein the integration unit is configured to generate a QR-code such that it has the source indication and the time stamp in coded form.

4. The apparatus according to claim 3, wherein the integration unit is configured to code the generated QR-code into an area of the extended image data non-visible for an user, in particular into a black-level area of the extended image data.

5. The apparatus according to claim 3 or 4, wherein the integration unit is configured to additionally code a hash value cal culated from the image data into the QR-code.

6. The apparatus according to one of claims 1 to 5, wherein the respective camera device has a camera monitoring device which com prises the integration unit and is connected via a respective image transmission interface, in particular via Camera Link, with the camera and the transmission channel.

7. The apparatus according to one of claims 1 to 6, wherein the respective screen is coupled with the transmission channel via a screen monitoring device, wherein the screen monitoring device is connected via a respective digital image transmission interface, in particular via Digital Video Interface, with the screen and with the transmission channel.

8. The apparatus according claim 7, wherein the screen monitoring device is configured to extract the source indica tion and the time stamp from the extended image data transmitted via the trans mission channel and to transmit the extracted source indication and the ex tracted time stamp via the interface to the control apparatus.

9. The apparatus according to one of claims 6 to 8, wherein the screen monitoring device is configured to perform an image examina tion regarding a freeze image supplied by the camera or cyclic images in the ex tended image data by means of a comparison of hash values transmitted by the extended image data and/or in dependence on a Fast Fourier transformation.

10. The apparatus according to claim 8 or 9, wherein the screen monitoring device is configured to fade in an overlay or a black image on the screen if the validation indicates a concealment of the corre sponding camera.

11. The apparatus according to one of claims 8 to 10, wherein the control apparatus is configured to interlock a weapon couplable with the apparatus if the validation indicates a concealment of a relevant image.

12. The apparatus according to one of claims 8 to11, wherein the screen monitoring device is configured to mark the transmitted im age data as freeze image if the validation indicates a display of a freeze image or a display of a cyclic image or a display of a strongly delayed image.

13. The apparatus according to one of claims 8 to 12, wherein the control apparatus is configured to suppress a forwarding of the transmitted extended image data if the validation indicates a display of a freeze image or a display of a cyclic image or a display of a strongly delayed image.

14. The apparatus according to one of claims 8 to 13, wherein the control apparatus is configured to interlock a weapon couplable with the apparatus if the validation indicates a display of a freeze image or a display of a cyclic image or a display of a strongly delayed image.

15. The apparatus according to one of claims 8 to 14, wherein the control apparatus is configured to control, in dependence on the vali dation, a secure failure display which indicates a failure of one or more of the N2 camera devices.

16. The apparatus according to claim 15, wherein the secure failure display comprises a background illumination of the screen, a dedicated display element, for example, a LED, arranged at the screen and/or a touch foil arranged at the screen.

17. A vehicle, in particular military vehicle, with an apparatus for validatable output of images according to any of claims 1 to 16, wherein the N2 camera de vices are arranged at an outer skin of the vehicle and the N1 screens are ar ranged in a room of the vehicle.