Ongoing incident
Incident Report for Aiven
Resolved
This incident has been resolved.
Posted Jun 29, 2021 - 05:05 UTC
Update
Our team is continuing to roll out a permanent fix to all affected systems
Posted Jun 29, 2021 - 01:46 UTC
Update
Our team is continuing to roll out a permanent fix to all affected systems.
Posted Jun 28, 2021 - 22:46 UTC
Update
We are continuing to roll out a permanent fix to all affected systems.
Posted Jun 28, 2021 - 21:27 UTC
Update
All systems have been patched. We are gradually rolling out the permanent fix to all affected systems.
Posted Jun 28, 2021 - 18:46 UTC
Update
We have applied fixes for all new nodes and services. A temporary fix has been applied to existing nodes. We are in the process of applying a permanent fix for the replication issue.
Posted Jun 28, 2021 - 16:29 UTC
Update
We have applied fix for the root cause and no new services or nodes should be affected by this. A temporary fix has been applied to almost all previously affected nodes to fix any immediate replication issues. We are still working on applying a permanent fix for all affected nodes.
Posted Jun 28, 2021 - 15:21 UTC
Update
We have identified the root cause of the problem and we have prepared a fix, which is ready to be deployed. At the same time we are still making the necessary corrections on the affected nodes in order to restore normality.
Posted Jun 28, 2021 - 14:07 UTC
Identified
PostgreSQL replica nodes fail to connect to new master after failover. We are manually fixing affected services to fix any immediate issues while working on fixing the root cause. There may be higher than expected replication lag for standby and separate read-replica nodes while the issue is ongoing.
Posted Jun 28, 2021 - 11:53 UTC