Descriptors cache¶
Matching a face by a list means reading descriptors of all faces of the list. The descriptors cache keeps them in memory, so such matching is done without requests to the luna-faces database.
The cache is filled with face descriptors of the default descriptor version linked to the cached lists. It is used for matching requests with the list_id candidate filter. If the list is not in the cache, the request is matched in the database as usual.
Enable¶
The cache is enabled by the cache_enabled field of the DESCRIPTORS_CACHE setting and requires luna-faces usage (see the ADDITIONAL_SERVICES_USAGE setting). The service runs the cache in a separate process shared by all the service workers: the descriptors are stored once whatever the worker count is, and the workers send their matching requests to this process, see rpc_settings.
The service starts to use the cache when all descriptors are loaded into it. Until then, and always for the lists which are not cached, matching requests are performed in the database.
Configuration¶
The DESCRIPTORS_CACHE setting:
cache_enabled - whether the cache is used
updating_cache_interval - interval between cache updates, in seconds
matching_settings - thread_count of matching threads (0 - by the count of CPU cores), tasks_count and batch_size of the queue which batches matching requests
rpc_settings - timeouts and the pool size of the connection between the service and the cache process
cached_data.faces_lists - include list ids to cache (null - all the lists), exclude list ids to skip and large_list_batch_threshold - a face count in a list from which matching requests are batched
Environment variables:
VL_CACHED_MATCHER_KEYS_RETENTION - how long, in seconds, the cache keeps keys of the applied changes, 2 seconds by default. See Synchronization.
VL_MATCHERLIB_BATCH_SIZE - size of a batch (chunk) of descriptors in the cache storage, the matcherlib default if not set. A larger batch may speed up matching of large lists.
Note
Changing DESCRIPTORS_CACHE restarts the cache process, so the cache is loaded again.
How it works¶
Loading¶
At start the cache remembers the maximum link key, unlink key and list deletion id of the changes which were made earlier than VL_CACHED_MATCHER_KEYS_RETENTION seconds ago: the transactions of such changes are expected to be committed by that moment, so the loading sees all of them. Then the cache loads all the linked faces, by several connections in parallel, by ranges of face ids. Everything committed later, including the changes made during the loading, has greater keys and is applied by the first synchronization.
Synchronization¶
Every updating_cache_interval seconds the cache applies the changes made in the database in the following order: removed lists, unlinked faces, linked faces.
Every kind of the changes is tracked by its own database sequence: list deletion ids, unlink keys and link keys. For each sequence the cache reads the keys greater than its safe key and applies only the keys which are not applied yet, so a change is never applied twice. Descriptors are read from the database only for the new link keys.
A transaction takes a key from a sequence when it inserts a row, but the row becomes visible to other transactions only after the commit, so a smaller key may appear in the database later than a greater one. To not lose such changes, the applied keys are kept for VL_CACHED_MATCHER_KEYS_RETENTION seconds and the keys of this period are read on every update. The safe key moves only to the keys applied earlier than the retention: by that moment these keys and all smaller ones are committed.
If an update fails, for example the database is unavailable, the safe keys are not moved and the same changes are applied by the next update.
Warning
A change of a transaction which is committed later than VL_CACHED_MATCHER_KEYS_RETENTION seconds after its start does not get into the cache until the service is restarted. The value should exceed the duration of the longest writing transaction of luna-faces and, if the database is replicated, the replication lag.
Matching¶
A matching request with the list_id filter is performed by the cache. Requests are batched to use the matching threads better, see matching_settings. If the list is not in the cache, for example it is excluded or not loaded yet, the request is performed in the database.
A face linked to a list appears in matching results within updating_cache_interval seconds after the commit of the transaction which has linked it.