Colossus File Format - old

Owned by Guy Avery (Deactivated)
Last updated: Sept 14, 2021 by Charlotte Dakin-Norris
3 min read

When using the Colossus recording tool it will connect to a radar and write the raw radar to a simple binary file. This can be used for file playback or importing into another software application. This document explains the format of the file. One point to note is that tool interprets the messages coming from the radar and records them to the file in the same format. Each message type and its structure is covered in the Colossus Network Protocol documentation. 

For information on recording and playing back data using the Colossus recording tool, please refer to Colossus Record and Playback Tool in the Witness 4.0 documentation.

On this page:

Overview

When the recording tool connects to the radar it will receive a radar configuration - this is always the first record to be saved in the data file. The tool then requests that the radar starts sending FFT data. Each packet of FFT data (equates to an azimuth of data)  is then added to the file. Depending on the radar configuration, each azimuth can contain FFT data of variable length. Reducing the amount of data we transmit from the radar can significantly lower network bandwidth and therefore significantly reduce the size of the recording file. For ease, each file record is prefixed with a total message size, message type  and time stamp represented by elapsed system ticks.

When several files are presented as a continuous record each file will start with an un-compressed configuration record in network byte order, this allows software to start playback from any file within a much longer sequence.

File Structure

As each message is received from the radar it is recorded to the binary file as File Record. Each record contains a simple header, and the original message payload.

Colossus Protocol
Please note that the structure of each message within the File Record is not covered here - they are all documented in the Colossus Network Protocol documentation. All the messages are stored in network order so they match the same structure as outlined in the network protocol. 

All records are streamed to the file in order they arrive over the network. To read the data back, applications will need to scan through the records and read in the record size so it can calculate the length of each records.

File Record Structure

Each record contains a fixed size header and a payload (Note this header contains little-endian numbers, the payload will contain big-endian). The payload is the network message received from the radar.

File Record Header

Field

Type [Size]

Description

Field

Type [Size]

Description

Length

int_32t [4]

The size of the message data. Therefore total record size is Record Header Size + Length. The header has a fixed size of 13 bytes.

Type

uint_8t [1]

The type of message in the message data. See TCP Message Types.

Top

uint_32t [4]

Upper byte of time, represented as total number of ticks

Bottom

uint_32t [4]

Lower byte of time, represented as total number of ticks

File Record Data

Field

Type [Size]

Description

Field

Type [Size]

Description

Message Data

uint_8t[Length]

Message data represented as an array of bytes. See TCP Networking for information on each message type. The total length of this payload is provided as the first field in the header.

 

Related information