NovaCloud-Hosting - Notice history

Website - Operational

100% - uptime
May 2025 · 99.85%Jun · 100.0%Jul · 100.0%
May 2025
Jun 2025
Jul 2025

Dashboard - Operational

100% - uptime
May 2025 · 99.85%Jun · 100.0%Jul · 100.0%
May 2025
Jun 2025
Jul 2025
Operational

Dedicated-Server EYG1 - Operational

Dedicated-Server Management - Operational

FFM1 IP-Transit - Operational

100% - uptime
May 2025 · 100.0%Jun · 100.0%Jul · 99.55%
May 2025
Jun 2025
Jul 2025

Notice history

Jul 2025

Multiple RYZEN VHOSTs + Dedicated servers unreachable
  • Resolved
    Resolved
    This incident has been resolved.
  • Postmortem
    Postmortem

    Summary,

    Between 20:29 and 20:32 GMT+2, several systems became temporarily unreachable due to a power distribution issue at the datacenter. The incident impacted specific power feeds and affected servers equipped with only a single power supply unit (PSU).

    Affected Systems,

    • Virtual Servers: RYZEN-VHOST 01, 03, 04, 05,

    • Dedicated Servers: A small number of systems with single-PSU configurations,

    • Unaffected: The majority of our infrastructure remained fully operational, including all dual-PSU systems,

    Root Cause,

    The incident was caused by a fault in one of the datacenter’s internal power subdistribution units. This resulted in a brief power loss on a specific feed.

    Resolution,

    Power was automatically restored by the datacenter within less than a minute. All affected systems came back online shortly afterward. We have confirmed that no data loss occurred.

    We are sorry for any inconvenience caused by this incident. Please open a ticket immediately if you still experience an outage, so we can address the issue.

  • Identified
    Identified

    There was a powerloss on one of the power feeds. It seems that the power has been restored within a few minutes and we are awaiting a statement from the datacenter to what has been affected and why.

    All affected Dedicated Servers and Root-Servers have already been started.

  • Investigating
    Investigating

    We are currently investigating this incident.

IP-Transit FFM1 - System replacement and upgrades
  • Completed
    23 July, 2025 at 9:59 PM
    Completed
    23 July, 2025 at 9:59 PM

    Maintenance has completed successfully.

    Since July 23, 2025, the setup has been successfully operating in active-active mode on both routers.

  • Update
    20 July, 2025 at 7:09 AM
    In progress
    20 July, 2025 at 7:09 AM

    even if everything is currently running, the measures are not yet fully completed. we will leave this window open until all work is completed so that parallel operation of the routers works without any problems. at the moment, however, gretap, vxlan, wireguarda and gre should work until we continue later.

  • Update
    20 July, 2025 at 4:23 AM
    In progress
    20 July, 2025 at 4:23 AM

    ok seems like that we have some issue going on after some runtime, looking into it...

  • Update
    20 July, 2025 at 4:04 AM
    In progress
    20 July, 2025 at 4:04 AM

    since about 06:04 the load is already fully active on the new routers, please open a ticket if you encounter problems, currently everything seems to have worked as planned.

  • In progress
    20 July, 2025 at 4:00 AM
    In progress
    20 July, 2025 at 4:00 AM
    Maintenance is now in progress
  • Planned
    20 July, 2025 at 4:00 AM
    Planned
    20 July, 2025 at 4:00 AM

    We are planning for a scheduled maintenance during that time.

    This is our response to repeated minor interruptions since the migration from FRA1 to FFM1. Specifically, we will:

    - Increase the total processing bandwidth from 20 Gbps to 80 Gbps, i.e. quadruple it.

    - Replace the currently heavily loaded system with 2 new future-proof systems

    - Switchover to a BGP multipath setup for consistent redundancy and maximum reliability

    For affected customers, this means that there may be several outages during the maintenance-window until the work has been completed.

    We apologize for any inconvenience caused.

router1.ffm1 is down
  • Resolved
    Resolved
    is back up. This incident was automatically resolved by Instatus monitoring.
  • Postmortem
    Postmortem

    Summary

    On July 17th, 2025, starting around 05:30 GMT+2, we experienced a partial connectivity issue affecting traffic to the endpoint 5.83.150.1. A portion of external networks was unable to reach some prefixes including 5.83.150.0/24, while others remained fully functional.

    Initial observations pointed towards a potential BGP routing problem, but external route propagation appeared fully intact across all monitors.

    Root Cause

    The issue was caused by a malfunctioning SmartMitigate DDoS filtering node within the upstream provider's infrastructure. The affected host had stopped forwarding traffic.

    As a result, incoming traffic that was routed through this specific mitigation node was effectively dropped, leading to asymmetric reachability depending on which SmartMitigate host was assigned.

    This incident was not related to our own BGP announcements or configurations.

    Resolution

    The upstream provider identified the faulty SmartMitigate host and removed it from operation. Immediately afterward, traffic flows normalized and full connectivity to the affected prefix was restored. This occurred around 09:20 GMT+2.

    Impact Overview

    • Affected Prefix: 5.83.150.0/24

    • Root Cause: Faulty SmartMitigate (DDoS filter) node

    • BGP Status: Unaffected – all routes remained correctly announced and visible

    • Duration: ~3 hours 50 minutes

    • Impact: Partial traffic loss (limited to traffic routed via the affected SmartMitigate node)

    Current Status

    Connectivity has been fully restored. The root cause has been addressed at the upstream provider level. No further disruptions are expected related to this issue.

  • Identified
    Identified
    We are currently investigating a potential issue with our BGP announcements. Based on external reachability tests, it seems that a significant portion of the internet is unable to reach our endpoint, suggesting that traffic is not being properly propagated or accepted by certain upstream networks.
  • Investigating
    Investigating
    is down at the moment. This incident was automatically created by Instatus monitoring.

May 2025

Network Migration: IP-Transit FRA1 -> FFM1
  • Completed
    01 June, 2025 at 1:22 AM
    Completed
    01 June, 2025 at 1:22 AM

    Maintenance has completed successfully.

    Since everything has been running for over 15 hours, the maintenance is considered complete. If you register any problems, please open a ticket.

  • Update
    31 May, 2025 at 10:16 AM
    In progress
    31 May, 2025 at 10:16 AM

    Our IPv6 Endpoint is also working since 12:16 finally, there was some error with the prefix. we will apply last optimizations to the automation during the day.

  • Update
    31 May, 2025 at 12:30 AM
    In progress
    31 May, 2025 at 12:30 AM

    other Updates:

    • Announcement of some prefixes did not pass, fixed since 00:18

    • IPv6 NDP Issues due to Large Amount of IPs and broken local-link, fixed since ~02:25

    • Wireguard fixed

  • Update
    30 May, 2025 at 9:46 PM
    In progress
    30 May, 2025 at 9:46 PM

    Most prefixes are back since 23:46.

  • In progress
    30 May, 2025 at 8:45 PM
    In progress
    30 May, 2025 at 8:45 PM
    Maintenance is now in progress
  • Planned
    30 May, 2025 at 8:45 PM
    Planned
    30 May, 2025 at 8:45 PM

    As all IP-Transit customers have already received an e-mail about, we will completely migrate our IP-Transit FRA1 infrastructure to a new Location FFM1 and new Routers. Details are in that e-mail.

    Please note that there may be several shorter and longer outages during this period. However, we do not expect any major downtimes.

    After the planned disannounce of all prefixes at 23:00 we will announce all prefixes at the new location in a timely manner.

    After that, once a connection has been established for the first time, no further outages are to be expected until the end of the maintenance window.

    As soon as the announcements at the new location have been started, we will post it here.

    It can take up to 24 hours for all routers out there to accept the new routes, but in most cases this will be much shorter, like some minutes to 1 hour (depends on your ISP and their Upstreams).

    All precautions have been taken on the upstream side so that the filters allow the new announcements immediately.

    If you think that something is not working as it should after the changeover, please do not hesitate to open a ticket.

May 2025 to Jul 2025

Next