Earthling - Notice history

This incident has been resolved.

DC Update:

"Our onsite team is currently bringing our UPS Systems online. We have our UPS vendor onsite assisting with this as we bring up UPS-4 and then UPS-R after that.

As these systems are brought online, we will concurrently work on bringing carriers up.

We are currently at 30% for completion toward bringing the site back online and the revised ETA for bringing up the critical infrastructure systems is approximately 6 hours. We are still planning for an evening time frame when clients will be able to come back on site."

The EWR Secaucus data centre remains powered down at this time per the fire marshal, resulting in our servers being offline. We want to apologise for the downtime. This is out of our hands and we are unable to access any of our servers due to the outage, you may keep up-to-date with their investigation and updates here: https://status.royalehosting.net/incident/231234

We are continue to monitor the status regarding our providers providers datacenter, if you haven’t read, a fire has broke out local to the data centres. It is something that nor us or Pixelnest (our main providers) can solve.

Our bots will come back on automatically once servers are back online.

This incident has resulted in our bots offline again. This is something out of our reach, you may view their status here: https://status.royalehosting.net/incident/231234

This incident has been resolved.

UPDATE 8TH AUGUST 2023: We have since switched to another host and fixed the issues listed below.

We have attempted to put Earthling back online. We know that a few things are a bit buggy, like the "Add to Server" button on the main profile (which results in iOS devices crashing); we are still waiting for Discord to fix this. You may now use Earthling like usual and invite it back using this link: https://earthlingbot.net/invite

I appreciate your patience, and we hope that you will be able to use us again easily.
If you have any problems, please open a ticket, and we will be happy to assist you as quickly as possible.

We continue to wait for Discord.

We continue to work with Discord.

We continue working with Discord; they are having problems resolving the error; we hope they can fix it soon.

We continue working with Discord to resolve our problems before putting Earthling back online. We have marked this incident as "Under Maintenance" while repairing our systems.

Our Support Bot on our Support server is also now back online.

We continue working with Discord to resolve our problems before putting Earthling back online.

Our Support Bot on our Support server is also now back online.

We are still waiting for Discord to fix some issues before we return Earthling online. However, we have been monitoring Canary for the past few hours and have deemed it safe again.

We have started to work on our recovery plan. We have enabled Canary bot and will be monitoring it.

We have been compromised again; Earthling has left all servers. We have reset the token again.

We have put Earthling back online after securing and resetting our tokens. We will monitor the bot closely for anything odd that may happen and take immediate reaction.

We have taken as many steps as quickly as possible to secure all data and servers. We have reset all passwords and tokens in case - all user data is safely secure.

We are so sorry for our service in the past few days, including today.

We are unsure if this was an attack towards Earthling or our hosting provider as a whole just yet; however, we have taken steps and have ensured the hackers do not have access to our bot.

We appreciate your patience; we believe this issue is finally resolved.

We will take extra steps with our provider to prevent this from happening in the future. But again, we are sorry for all disruptions we may have caused. We will continue to monitor the bot over the next few days.

Thank you all for using us; it means so much! <3

A fix has been implemented, and all our services are returning online!

Again, we are sorry for any inconvenience caused and will continue to monitor the situation.

We don't want to leave you in the dark! We are still highly sorry for the downtime over the past day. Although we have been close to the owner of our hosting provider to get as many updates as possible, we believe the bot will be up very soon.

They are waiting for their new IP addresses to be propagated, allowing all services to return online. If you have any questions or issues, please join our support server. This is the most significant outage we've had in Earthling's lifetime; we hope our providers take steps to prevent this from happening again.

We will post any major updates here as quickly as possible.

Our host has informed us that they will be soon assigning us to a new IPv4, allowing our bots to come back online. This should be done some point within the next few hours.

Our services continue to be offline.

We no longer have an estimated time for when this will be resolved.

We are extremely sorry for the downtime of our service.

Due to performance issues, we have turned off the secondary server hosting Earthling - the bot continuously failed to respond to interactions due to the network connection not being enough to deliver at the rate requests were coming in.

We hope to get the central server up within 30-60 minutes.

We have exported and loaded up Earthling on our secondary machine for the time being whilst we wait for our host to fix their services.

The bot may be slower than usual during this time.

We continue monitoring our servers to get them back online quickly. In addition, our provider is working on a complete system restoration to get all bots on their site back online.

You may view the status of their situation here: https://status.pixelnest.gg/incidents/v5b8d3qthf4p

We once again apologise for all the disruption.

We would like to apologise for what is now the most extended outage we have had in months, causing all our services to go offline. Unfortunately, this incident is out of our control, and we cannot fix it. We are working with our providers to resolve the issue as quickly as possible.

We have identified the cause and are looking for a solution.