Bazefield Tag Delete Jobs¶

The JobBazeTagDelete class is a job instance that performs the deletion of Bazefield tags from Bazefield historian and database. This class is defined in the job_instance_baze_tag_delete.py file and inherits from the JobInstance base class.

Python Class¶

Required Parameters¶

All the parameters listed below must be provided in the parameters column of the job_instance table in JSON format.

object_filters: A dictionary with filters to apply when selecting objects to delete tags from. It has two keys: "ignored" and "selected". Each key can contain any of the following keys: "types", "models", "instances". The values are lists of strings. For example:
```
{
  "selected": {
    "types": ["Turbine"],
    "models": ["GE103-1.X"]
  },
  "ignored": {
    "instances": ["CLE-CLE1-01"]
  }
}
```
This example selects all objects of type "Turbine" and model "GE103-1.X", except the object with instance name "CLE-CLE1-01".
tag_filters: A dictionary similar to object_filters, but for tags. It has two keys: "ignored" and "selected". Each key is a list of tag names to ignore or select. For example:
```
{
  "selected": ["ActivePower", "WindSpeed"],
  "ignored": ["WindSpeed"]
}
```
This example selects "ActivePower" and "WindSpeed" tags, but then ignores "WindSpeed", resulting in only "ActivePower" being selected. The ignored filter is more useful when regex is used.
regex_tags: Boolean flag. If True, the tag names in the tag_filters will be treated as regex patterns. Default is False.
start: The start datetime of the deletion period in the format YYYY-MM-DD HH:MM:SS.
end: The end datetime of the deletion period in the format YYYY-MM-DD HH:MM:SS.

The job will also directly lookup the following parameters from the settings table in the database:

job_step_interval: The interval in seconds between job steps. If not set, defaults to 0 seconds. This is really important as Bazefield historian can stop responding if too many requests are sent in a short period of time.
request_subperiod_days: The number of days to split the deletion period into smaller subperiods. If not set, defaults to 100 days. This is important to avoid timeouts when deleting large periods of data.

Job Steps¶

The job steps are created for each object instance. For each object instance, a job step is created that lists all the tags that will be deleted for that object. The steps are ordered by object model name and then by object instance name.

Automatic Processing¶

Job Creation¶

All jobs are created periodically by the job-creator-bazetagdelete Airflow DAG. The DAG will create one job instance for each object model using as base the following settings in the database (all under the baze_tag_persistance section):

object_filters: Defines the object filters to use when creating the jobs. This is a dictionary with two keys: "ignored" and "selected". Each key can contain any of the following keys: "types", "models", "instances". The values are lists of strings. For example:
```
{
  "selected": {
    "types": ["Turbine"]
  },
  "ignored": {
    "instances": ["CLE-CLE1-01"]
  }
}
```
This example selects all objects of type "Turbine", except the object with instance name "CLE-CLE1-01".
tag_regex_patterns: A list of regex patterns to use when selecting tags to delete. This is a dictionary with two keys: "ignored" and "selected". Each key is a list of regex patterns. For example:
```
{
  "selected": ["^Wind.*", "^ActivePower$"],
  "ignored": ["^WindSpeed$"]
}
```
This example selects all tags that start with "Wind" and the tag "ActivePower", but then ignores the tag "WindSpeed".
max_age_days: The maximum age of the data to delete in days. This is a dictionary where the keys are the object types and the values are the maximum age in days. For example:
```
{
  "Turbine": 30,
  "WeatherStation": 60
}
```
This example deletes data older than 30 days for objects of type "Turbine" and older than 60 days for objects of type "WeatherStation".
job_deletion_days: The number of day a delete job will cover, so the start of the job will be max_age_days - job_deletion_days from now. For example, if max_age_days is 30 and job_deletion_days is 7, the job will delete data from 37 days ago to 30 days ago.

Job Execution¶

All jobs are executed periodically by the job-processor-bazetagdelete Airflow DAG. The DAG will look for all pending jobs and execute them one by one. The job will be marked as completed when all job steps are completed.