NovaCloud-Hosting - Notice history

All systems operational

Website - Operational

100% - uptime
Mar 2025 · 99.78%Apr · 100.0%May · 99.85%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Dashboard - Operational

100% - uptime
Mar 2025 · 99.78%Apr · 100.0%May · 99.85%

Previous page

Next page

Mar 2025
Apr 2025
May 2025
Operational

Game-Panel - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Dedicated-Server Management - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025
Operational

Dedicated-Server EYG1 - Operational

Operational

Dedicated-Server NBG1 - Operational

FFM2 Datacenter (NTT-FRA1) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Notice history

May 2025

Network Migration: IP-Transit FRA1 -> FFM1
  • Completed
    01 June, 2025 at 1:22 AMUTC
    Completed
    01 June, 2025 at 1:22 AMUTC

    Maintenance has completed successfully.

    Since everything has been running for over 15 hours, the maintenance is considered complete. If you register any problems, please open a ticket.

  • Update
    31 May, 2025 at 10:16 AMUTC
    Update
    31 May, 2025 at 10:16 AMUTC

    Our IPv6 Endpoint is also working since 12:16 finally, there was some error with the prefix. we will apply last optimizations to the automation during the day.

  • Update
    31 May, 2025 at 12:30 AMUTC
    Update
    31 May, 2025 at 12:30 AMUTC

    other Updates:

    • Announcement of some prefixes did not pass, fixed since 00:18

    • IPv6 NDP Issues due to Large Amount of IPs and broken local-link, fixed since ~02:25

    • Wireguard fixed

  • Update
    30 May, 2025 at 9:46 PMUTC
    Update
    30 May, 2025 at 9:46 PMUTC

    Most prefixes are back since 23:46.

  • In progress
    30 May, 2025 at 8:45 PMUTC
    In progress
    30 May, 2025 at 8:45 PMUTC
    Maintenance is now in progress
  • Planned
    30 May, 2025 at 8:45 PMUTC
    Planned
    30 May, 2025 at 8:45 PMUTC

    As all IP-Transit customers have already received an e-mail about, we will completely migrate our IP-Transit FRA1 infrastructure to a new Location FFM1 and new Routers. Details are in that e-mail.

    Please note that there may be several shorter and longer outages during this period. However, we do not expect any major downtimes.

    After the planned disannounce of all prefixes at 23:00 we will announce all prefixes at the new location in a timely manner.

    After that, once a connection has been established for the first time, no further outages are to be expected until the end of the maintenance window.

    As soon as the announcements at the new location have been started, we will post it here.

    It can take up to 24 hours for all routers out there to accept the new routes, but in most cases this will be much shorter, like some minutes to 1 hour (depends on your ISP and their Upstreams).

    All precautions have been taken on the upstream side so that the filters allow the new announcements immediately.

    If you think that something is not working as it should after the changeover, please do not hesitate to open a ticket.

Customer Dashboard maintenance - investigating backend issue
  • Completed
    29 May, 2025 at 9:05 PMUTC
    Completed
    29 May, 2025 at 9:05 PMUTC
    Maintenance has completed successfully
  • In progress
    29 May, 2025 at 8:05 PMUTC
    In progress
    29 May, 2025 at 8:05 PMUTC
    Maintenance is now in progress
  • Planned
    29 May, 2025 at 8:05 PMUTC
    Planned
    29 May, 2025 at 8:05 PMUTC

    We are performing an urgent maintenance to investigate an issue with the backend of the system, causing some small issues which need to be fixed.

IP-Transit FRA1 Outage
  • Resolved
    UTC
    Resolved

    This incident has been resolved.


    We are sorry for the short outage. the interruption and possible packetloss before is due to currently limited capacities, attacks and the associated higher susceptibility to errors, which will soon be corrected see other maintenance window. We do not expect any further major outages.

  • Monitoring
    UTC
    Monitoring

    We restored the network and currently monitoring it deeper.

  • Investigating
    UTC
    Investigating
    We are currently investigating this incident.
IP-Transit outage
  • Resolved
    UTC
    Resolved

    This incident has been resolved. A statement will follow later this day.

  • Investigating
    UTC
    Investigating
    We are currently investigating this incident.
IP-Transit Outage IPv6
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Investigating
    UTC
    Investigating
    We are currently investigating this incident.

Apr 2025

RYZEN-02-Game-Host outage
  • Monitoring
    UTC
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Identified
    UTC
    Identified

    The hostsystem is currently offline. The issue should be solved in a few minutes.

Root-Server management outage
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Identified
    UTC
    Identified

    Users are currently experiencing errors in our Dashboard for the Root-Server management. The issue is caused by a dependency runnin on the Intel Xeon Hostsystem which is currently undergoing maintenance work.

    No servers are unavailable, only the management or Root-Servers is currently not possible.

XEON-01-VHOST Maintenance
  • Completed
    28 April, 2025 at 10:38 PMUTC
    Completed
    28 April, 2025 at 10:38 PMUTC

    We have successfully changed to a reliable and fast Storage-Layout.

    You can now enjoy high performance as usual.

  • Update
    17 April, 2025 at 12:00 PMUTC
    Update
    17 April, 2025 at 12:00 PMUTC

    We are planning to further investigate the outage from this morning https://novacloud.instatus.com/cm9e4e72j00gp3g2rnkezwo0b

    In order to prevent data-loss, all servers will be stopped while this maintenance is conducted.

  • In progress
    16 April, 2025 at 6:05 PMUTC
    In progress
    16 April, 2025 at 6:05 PMUTC

    We are going to migrate all servers onto another hostsystem which we've deployed today. Thanks to the new hostsystem being an Intel Xeon with enough ressources, it won't be necessary to stop any servers. Therefore, we don't expect any significant outages caused by the maintenance. There may be a reboot of the servers, which shouldn't last longer than a few minutes.

  • Planned
    16 April, 2025 at 12:00 PMUTC
    Planned
    16 April, 2025 at 12:00 PMUTC

    The issue doesn't seem to be solved permanently and occurs from time to time, causing downtimes.

    We are planning a maintenance for 17.04.2025, where all VMs will be migrated onto another hostsystem temporarily in order to reconfigure the Storage layout of the Intel Xeon HDD Pool.

    All affected customers with Storage Servers will be awarded 5 days of runtime and all customers with Intel Xeon servers 2 days of runtime. Please open a ticket after the maintenance has finished in order to receive the compensation. All tickets opened before the maintenance has finished will be put on hold and processed once we have finished the migration.

    We are extremely sorry for the inconvenience caused by these incidents and hope you can keep your trust in us after we've finished this maintenance, which will solve the problems permanently in order to deliver the quality and reliability we always strive to achieve.

XEON-01-VHOST HDD-Pool inaccessible
  • Resolved
    UTC
    Resolved

    We have completed the maintenance and resolved the issue with a more reliable Storage-Layout.

    The setup is tested to ensure that this issue won't occur in the future. Thanks for your understanding.

    All affected customers (XEON Root-Servers + Standard Game-Servers + Storage Root-Servers) can open a ticket to request a free service extension of 5 days.

  • Monitoring
    UTC
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    UTC
    Identified

    We have made some changes and are currently rebooting the hostsystem in order to apply the changes.

    Additionally, the NVMe cache will be disabled to test whether this issue is caused by the caching method we are currently using.

  • Investigating
    UTC
    Investigating

    Any IO operations on the HDD pool are not possible again. We are currently investigating the issue and trying to get the HDD pool operational.

  • Monitoring
    UTC
    Monitoring

    A routine Raid-Check resulted in a bug which locked up read/write operations.

    We have managed to restore normal operation, but with degraded speed of the HDD-Pool in order to prevent another issue. As soon as we have found a solution, full speed will be restored.

    MAINTENANCE: https://novacloud.instatus.com/cm9e9zh6o00nl79u2h3ibnqhn

  • Investigating
    UTC
    Investigating

    Some servers cannot be started and the Disk-IO of the HDD-pool is very low. We are currently investigating the cause for this issue.

Degraded Performance / Packetloss IP-Transit FRA1
  • Resolved
    UTC
    Resolved

    This case is hereby closed.

    Although no further measures have been taken to prevent this from happening again, this will be dealt with and fixed at the latest with the next major update we are currently working on, if possible.

  • Monitoring
    UTC
    Monitoring

    We implemented a fix and are currently monitoring the result.

    We have narrowed down the problem and reduced it to the details. Further details are being clarified in the background, the load is back to normal and everything is running smoothly. We apologize for the circumstances and strive to prevent this from happening again.

  • Update
    UTC
    Update

    We were able to narrow down the problem further by suspending a customer, but the exact cause is not yet clear and is now being analyzed while the network is more stable.

  • Investigating
    UTC
    Investigating
    We are currently investigating this incident.

Mar 2025

RYZEN-02-VHOST outage
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    The mainboard has been replaced and all servers are running.

  • Identified
    UTC
    Identified

    The mainboard seems to be faulty. This server still uses an ASRockRack mainboard, which aren't as reliable. A technician has been informed and will replace it with a new Supermicro mainboard.

    ETA: 30-45 Minutes

  • Investigating
    UTC
    Investigating
    We are currently investigating this incident.
FRA1 IPv6 Outage
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    We implemented a fix and are currently monitoring the result.

    In this case, it only seemed to specifically affect IP transit due to the network changes that were made and the resulting change in MAC address of the gateway. After rebuilding the NDP entrys it works again. We apologize for this. the incident will be closed soon.

  • Investigating
    UTC
    Investigating

    We are currently investigating this incident.

    Probably due to adjustments made by our ISP, ipv6 is currently no longer working.

Migration: Intel XEON Root-Servers to new XEON-01 host
  • Completed
    27 March, 2025 at 10:00 PMUTC
    Completed
    27 March, 2025 at 10:00 PMUTC
    Maintenance has completed successfully
  • In progress
    27 March, 2025 at 4:00 PMUTC
    In progress
    27 March, 2025 at 4:00 PMUTC
    Maintenance is now in progress
  • Planned
    27 March, 2025 at 4:00 PMUTC
    Planned
    27 March, 2025 at 4:00 PMUTC

    We are planning to migrate all Intel XEON servers to our new Intel Xeon E5-2697v4 hostsystem.

    Due to the cpu-architecture changing, we need to make an offline-migration which will result in a downtime of around 5-120 minutes depending on the disk-size.

EMERGENCY REBOOT: XEON-01 Vhost issues with Disk IO
  • Resolved
    UTC
    Resolved

    After continous monitoring troughout the day, the system is running stable and had no further issues.

  • Monitoring
    UTC
    Monitoring

    We implemented a fix and are currently monitoring the result.

  • Investigating
    UTC
    Investigating

    We have received a report that the Disk IO seems to be stuck sometimes, which is probably resulted by too many open files. We have adjusted kernel parameters and need to reboot the hostsystem. We will continue to monitor all systems after the reboot.

    Also there was an issue with the deployment of new servers which was caused by the same problem.

EYG1 Backups temporarily disabled
  • Resolved
    UTC
    Resolved

    The backup-speed issue has been permanently resolved by integrating our new Backup server. Backups can be created and restored now.

  • Investigating
    UTC
    Investigating

    Due to ongoing network-issues with the backup server at our EYG1 location, we are temporarily disabling the creation and restoring of backups. If you need to create or restore a backup, please create a ticket so we can do it manually for you.

    We are currently waiting for a response from our ISP as it is a network-related issue. We expect to have a new Backup server ready at the end of next week which is located at EYG1, which will permanently exterminate these issues and should increase the backup-speeds.

Mar 2025 to May 2025

Next