v.5.27.0

Improvements:

  1. [LUNA-6488] System info’s stats (InfluxDB backend) are fetched with the InfluxDB client instead of a hand-made parsing of the raw CSV response. Errors (InfluxDB is unavailable, the query is rejected or fails during the execution) are logged with the actual reason instead of being silently reported as empty statistics.

  2. [LUNA-6488] System info’s image_size and face_detection_size histograms (ClickHouse backend) report images and faces higher than the max meaningful size as a single +Inf bucket, the same way as the InfluxDB backend does, instead of w_<width>_h_inf and h_inf buckets.

  3. [LUNA-6488] System info’s image_size and face_detection_size histograms are bucketed the same way on both backends, as the InfluxDB backend does: face detection heights are rounded up to 10 pixels (ClickHouse used 50 pixels) and images wider than the max meaningful size are counted in the +Inf bucket (ClickHouse counted them by their width). The ClickHouse buckets are changed by the monitoring database migration: the wide images are moved to the +Inf bucket for the whole history, face detection heights are recomputed only for the days the raw monitoring data still fully covers (according to the actual TTL of the raw monitoring tables), the earlier days keep the 50-pixel buckets.

  4. [LUNA-8842] The gc task for general events now accepts the remove_samples parameter (boolean, default false) and the targets parameter (paths of the event’s user-defined fields that store links to images to remove along with the event), see create gc task.

Bugs fixed:

  1. [LUNA-6488] System info’s stats (InfluxDB backend) counted the last day of every month in the next month, and on the first day of a month reported the previous month twice. Every day is counted in its own month now, the same way as on the ClickHouse backend.

  2. [LUNA-6488] System info’s stats (InfluxDB backend) with data for more than one month split estimators_performance_stats into separate entries for batch_size and execution_time and image_processing_stats into several entries per month.

  3. [LUNA-6488] System info’s estimators_stats (InfluxDB backend) reported the usages of every service as a separate entry and included zero counts. Usages are summed up over all the services and zero counts are skipped now, the same way as on the ClickHouse backend.

  4. [LUNA-6488] System info’s image_processing_stats (InfluxDB backend) omitted image_load_time, image_size, face_detection_size and face_descriptor_score fields without data instead of returning null/empty values as the specification requires.

  5. [LUNA-6488] System info’s image_size (InfluxDB backend) reported images from 9951 to 10000 pixels high under broken keys like w_101_h_0 and skipped images narrower than 50 pixels, face_detection_size (InfluxDB backend) counted faces from 9991 to 10000 pixels high in the +Inf bucket. The aggregation flux tasks are fixed. The tasks of an existing installation are updated when the monitoring is prepared by storages with the --ignore-integrity option, the data aggregated before keeps the previous keys.

  6. [LUNA-6488] System info’s face_detection_size (ClickHouse backend) counted faces exactly 10000 pixels high in the +Inf bucket, face_descriptor_score (ClickHouse backend) counted some scores on a bucket boundary (e.g. 0.3) in the next bucket (score_0.35) because of the single precision rounding. The buckets are fixed by the monitoring database migration and recomputed for the days the raw monitoring data still fully covers, the earlier days keep the previous buckets.

  7. [LUNA-6488] System info’s estimators_stats (ClickHouse backend) did not report the face occlusion estimator usages (face_occlusion_estimator_usages): they were not aggregated by the monitoring database. The monitoring database migration starts aggregating them the same way as on the InfluxDB backend and counts them for the days the raw monitoring data still fully covers.

Migration:

See a migration manual.

[LUNA-6488] See a migration manual.