RuleScheduleService db updates are not atomic and could lead to duplicate scheduled rule execution

Description

This should never happen due to the fact that our scheduler in service executes RUN_ONCE_PER_CLUSTER. However it seems that on instances with heavy load 2 nodes or 2 scheduler threads can call this at the same time.

We need to make sure that when we read the schedules we select them with forUpdate()
and that we write the new next_execution date within one transaction so that only one node will ever add a scheduled rule once and other nodes will re-read if they selected the same nodes.

Environment

None

Status

Assignee

Andreas Knecht

Reporter

Andreas Knecht

Labels

Links

1

Fix versions

Priority

Configure