Classification Configs

Introduction

This section covers Classification Configuration and how to access the data it provides.

Overview

In order to become a track, an object needs to exhibit various attributes and behaviours, e.g. physical size, speed, directional change rate etc. Classification assumes different tracks would exhibit a certain set of attributes, and thus would be clearly different from other track types (see ATI.)

For example, people are smaller than vehicles. Vehicles travel faster than people. People can make quicker turns than vehicles, and so on. These attributes lead to the creation of a classification model. A model is created for each possible object type. The model lists likely values for each attribute (both a minimum and maximum, giving a range, or just a minimum). Each track generated within Witness 4.0™ is ‘scored’ against this model using those values.

This score is calculated by determining if certain attributes are within a specified range. For each attribute that is within the range, that model gets 1 attribute point. Each attribute point is then weighted according to how important that attribute is to the model. This gives each track a score. The default classification uses 8 tracks (or samples), so a track’s final score is given as a percentage of achieved score against possible total score (over those 8 tracks).

Whichever model ultimately scores higher is deemed the correct classification for that track.

Classification Config

This is essentially the folder that encompasses all the classification models. Multiple folders can exist, each with different configurations for the models they contain.

Classification Name: The name of this set of classification configurations. There will be a default folder named 'Default'.

Samples: The track count used to create a mean track final score before the track is classified.

Min. Probability Required (%): This is the minimum level of confidence required before you consider the track to have been successfully classified. If this confidence level is not met then you can specify that the system continues for a further n sightings. The value of n is supplied by the next parameter, Extra Sample Increment.

Extra Sample Increment: This value is used in conjunction with Min. Probability Required. If the probability level is not met then the system will continue to classify for a limited number of additional sightings. This number is specified by the Extra Sample Increment value. If at the end of this additional sample period the classification has a confidence level higher than Min. Probability Required then the track will be successfully classified. However, if it falls below this threshold then the track will remain unclassified for as long as its tracked. The default Extra Sample Increment value is 4, giving a potential sample period of 12 (8 + 4).

Be careful when increasing either Extra Sample Increment or the Minimum Probability Required because this could significantly delay the classification result. This may also have a knock-on effect for other parts of the system. For example, your rules. If you rely on classification in your rule configuration then you need to consider how to handle tracks which may be unclassified for a number of sightings. Note that there is an option in the rule configuration to handle unclassified tracks but this is not enabled by default.

Classification Models

The Classification models for tracks are:

It is not possible to achieve classification values for animals, and persons crawling. Unambiguous classification relies on models having attributes which are not the same as other models. Though a car may travel as slow as a person, or a person may change direction in some circumstances, with a turn rate the same as a car, size differentiates the two. Critically, it was discovered that an animal is indistinguishable from a crawling person using our track attributes. It is likely that a good model could be reached for an object name ‘animal or person crawling’, though of course this ambiguity is undesirable in a security system. Furthermore, the lowered threshold/plot extraction settings required to confidently detect these objects unfortunately leads to higher false alarms. To prioritise reliable classification, these classification models were therefore omitted.

Classification Properties

Each target (or track) is assessed based on properties provided by the radar and tracking software. These properties are defined by:

Range:

Min: This is the minimum value that the model is looking for from the summary sample data. If the summary data from the analysed track is greater than or equal to this value then it will have met the minimum criteria. Where there is no maximum value provided, see below, then this is the only criteria that needs to be met. If it is satisfied then the track is awarded the appropriate weighted points for this classification model, increasing the likelihood that the track is a match.
Max: This is the maximum value that the value is looking for from the summary sample data. If the summary data from the analysed track is less than or equal to this value then it will have met the maximum criteria. Note that this value is optional. If omitted then the model will assume any value above the minimum (see above) is a match. If a minimum and maximum are specified then both criteria must be met for the track to match this attribute of the model.

Summary Method: This specifies the method for summarising the recorded samples. All track sightings are processed, up to the sample count, and then each of the properties is summarised using this specified method before being compared against the model.

Min: This method means the lowest value seen on the track during the sampling period will be used to compare against the model.
Max: This methods means the highest value seen on the track during the sampling period will be used to compare against the model.
Mean: This method means the average of all values seen on the track during the sampling period will be used to compare against the model.
Truncated Mean: This method means that the average of all values (minus the outliers) seen on the track during the sampling period will be used to compare against the model. The truncated mean will sort the sample data and then remove a percentage of the values from the top and bottom of the sorted list. So the amount of samples ignored at the top and the bottom is % / 2. For example if you had sample data that looked like this 3,7,21,34,36,37,55,89 (8 samples) and the truncated percent was 25% then we would lose 2 of our 8 values (25%) but 1 from the top and 1 from the bottom. So the final value would be the average of the 6 remaining values (i.e. 7,21,34,36,37,55 - average is 190/6 = 31.7).

When using the Truncated Mean method you must also set the Truncated Percent value to indicate how many values at the top and bottom of the sample data you wish to ignore. Truncated Percent specifies how many outliers to ignore at the top and bottom of the sample data.

The properties are as follows:

Property	Description	Default Assessment
Size	The surface area of the target calculated using size in range x size in azimuth.	Does the (truncated) average size of the track across all sightings fall within the range of the model?
Speed	The speed of the track.	Does the maximum speed seen across all sightings fall within the range of the model?
Expected Speed	The typical speed expected of a track. This would normally be a smaller subset of the overall speed range. For example, a car can move as slowly as a walking person, however its expected speed would normally be much quicker. Similarly, whilst a vehicle may be able to travel at speeds from 0m/s to 44m/s (100 miles or 160km per hour) but in an urban road environment it’s more likely to be travelling from 4 to 14m/s (10mph to 30mph).	Does the (truncated) average speed of the track across all sightings fall within the expected speed range of the model?
Acceleration	This is the change in speed between track sightings.	Does the maximum acceleration across all sightings fall within the range of the model?
Weight	The total number of cells that make up the target.	Does the (truncated) average number of cells across all sightings fall within the range of the model?
Strength	Sum of the signal strength for each cell which makes up the target. Note this value is only truly effective if using "Input" or "Level Above" for the thresholding process. All processed data exits thresholding with a value of 255. Therefore, Strength, by default, is weight multiplied by 255.	Does the (truncated) average signal strength across all samples fall within the range of the model?
Seen Moving	This is the maximum speed that a track can travel before it is considered to be moving. This is used a base threshold to help distinguish between stationary debris and moving targets.	Does the average speed seen across all sightings exceed the value in the model?
Direction Change	The absolute change in direction from one sighting to the next. For example, a person may change direction more quickly and more drastically than a vehicle, and so would have a higher minimum value.	Does the average change in direction across all sightings fall within the range of the model?
Reset Trigger	A trigger which when its conditions are met will force the existing classification to be reset. This will result in the classification process starting again for this track. For example if a small, slow moving vehicle is incorrectly classified as a person you might want to force reclassification if the vehicle accelerates above normal walking speed. Clearly in this situation the track can't any longer be considered to be a person and reclassification should hopefully force it to change to a vehicle.

Each Classification Model by default is defined by these configurable properties except Debris, which by default is only concerned with Speed and Acceleration.

Category and Property Weighting

Each track property is group into a category. This is so each category can be weighted on a model by model to ensure the scoring system is biased towards the key attributes for a specific classification.

The categories are as follows:

Category	Default Properties
Speed and Acceleration	Expected Speed, Speed and Acceleration
Weight and Strength	Weight and Strength
Behaviour	Seen Moving and Direction Change
Size	Size

Each category and property is given a weighting. This can vary from model to model. These weightings will directly affect the model scoring. For example, if a target property matches a critical attribute then the score will be 50, whereas a matching optional property would only get 10 points.

Category	Score Weighting
Critical	x50
Desirable	x30
Optional	x10
Ignore	x0

Ignore effectively ‘turns off’ certain attributes from having an effect on classification.

Analysing Tracks in Playback

Each model can be fine-tuned by adjusting the properties. The key concept is to analyse tracks in Playback and review the typical values associated with each type (i.e. person vs vehicle) and then adjust the property values in the classification configuration to match.

This can be achieved by Querying under the Playback tab:

Set a Track Query (for any classification).
In the Results Panel, the query results will be tabled. One column is labelled 'History'.
Select the rewind icon in the History column for the track you wish to analyse.
A Track Details window will appear.
Select the Classification tab.
A table of results will appear, listing the track's classification probabilities. These are calculated as a fraction of achieved points out of total points possible e.g. 10500/17000

In the above table, the track has been classified as a Vehicle, but has in an equal probability (70.6%) of being classified as a Large Vehicle.

These results merit classification adjustment, in order to differentiate between Vehicle classification and Large Vehicle classification. By modifying the classification, you can make a new set of point values, alongside the original score, for comparison.

Classification Adjustment

Firstly, analyse the individual cell results by hovering over them with the mouse pointer. Small boxes of information regarding the individual values of the different properties will appear.
In this example, the Weight classification has been highlighted. Note that the Value has fallen below the Vehicle classification Weight range (45-65>31.88), and thus the track is smaller and therefore not a Large Vehicle.
Pull the Track Details window to one side, and select Classification Configs. Ensure the Config Function is enabled.
Select Edit, and then under Classification Models, adjust whichever value you see fit. In this example, the Vehicle: Weight and Strength: Weight range has been adjusted to 30-65.00.
When you select Save, a new tab will appear on the Track Details window: Modified Classification.
A change in the Weight range has changed the score of the Vehicle classification, and raised the probability of the track being a Vehicle to 76.5%, clearly differentiating it from a Large Vehicle.

There may be situations where a track doesn’t score highly at all in any model category. Regardless this track would still be classified according to the one with the highest score. To prevent this, the user is able to configure a percentage confidence rate to which the classification should aim, in the Advanced tab under the Configure Rules; https://navtechradar.atlassian.net/wiki/spaces/TUN/pages/211321151/ClearWay+Rules#Configuring-a-Rule window. You may set this to be very strict, however a satisfactory rate may be 70%

Creating a Classification Configuration

It may be necessary to create a new Classification Config for a system as the Default one is not editable. A new one can have it’s models settings tweaked and it is also possible to add / remove models.

To do this click, on Classification Configs in the Configuration Tree and select New:
Then select Classification Config:
This will open a Create New Classification Configuration wizard:
Give your new Classification Config an appropriate name and click Finish:
This will create a new Classification Config for you to modify. The Classification Models that were in the Default Classification Config have been replicated here, to allow you to tweak the settings, add or remove a model.
To edit a model, select it and navigate through the settings and amend the ones required:
By default, the Animal setting has not been enabled, so to enable it select the Enabled option:
To add another model, if it is similar to another in size and weight, it will be easier to clone one of the current models by clicking the clone icon:
Then select another Classification from the drop-down list:
This will then display the new model:
The settings can be adjusted to suit the new model:
To remove a model from the list, click on it and then select the trash icon:
Once you are happy with your modifications, click Save:
The new Classification Config will now be displayed in the Configuration Tree: