[1:45pm EST] The outages are fully resolved. We’re still waiting for more specific information from one of our network vendors, but the problems has been confirmed to be upstream network outages. We’ll continue to monitor and work internally to determine how we can prevent and/or minimize any future potential issues of similar nature. Please let us know at firstname.lastname@example.org if you have any questions.
[1:15pm EST] We are still experiencing problems for a small subset of our customers based in the US. Service for Canadian customers has been fully restored. The problem is related to network issues upstream by one of our 3rd party providers. The provider is working to both restore the outage and work on redirecting traffic in the meantime. We are hoping this will be resolved shortly and we will post more updates here as we know more. Unfortunately, no specific ETA is possible.
[12:30pm EST] We are currently experiencing performance problems and outages for some customers hosted in the US and Canada regions. Our team is investigating and working to resolve as quickly as possible. We will make ongoing updates here.
[9:21am Central] Full service has been restored. During the disruption, many customers had trouble viewing custom themes and logging in using authentication tools like Shibboleth. Please let us know at email@example.com if you notice any additional issues or have any questions.
[8:00am Central) We are currently experiencing problems for a few of our US-based customers with any custom themes, custom plugins, and authentication tools such as Shibboleth.
We have identified the problem and are in the process of fixing it. Full service should be restored soon and we will update here once that work is complete.
[4:21pm EST] Normal operations are returning. During the outage, many customers had trouble viewing and adding images and file uploads. Please let us know at firstname.lastname@example.org if you notice any additional issues or have any questions.
[1:41pm EST] We are seeing an increased number of errors and performance issues on some sites that we host. Sites still remain up and viewable on the web, but you may notice occasional errors with editing or updating a site and lagging performance. This is related to ongoing outages with Amazon Web Services, which we use for much of our infrastructure. You may continue to have problems loading sites, and in particular, adding or viewing images or file uploads. We are monitoring the situation and doing what we can to minimize impact. You can follow the Amazon outage at https://status.aws.amazon.com.
There is much news going around about a recent bug in Cloudflare which resulted in an extremely small number of Cloudflare customers’ that had the possibility of exposing customer data.
We are a Cloudflare hosting partner and do utilize Cloudflare services, but we are 100% confident that we were not impacted by this bug and that no customer data that we host was compromised for the following reasons:
- We’ve been assured by Cloudflare that none of the domains we host were impacted, including all CampusPress customers and our Edublogs.org network.
- We do not leverage the specific Cloudflare services that contained the originally disclosed bug.
- Even still, we decided to change our authentication keys and force all logged in users to re-login just to ensure no further impact was possible.
Please contact us at email@example.com should you have any further questions.
Resolved: [8:45 PM EST] We have confirmation that it is now fully resolved. This event was Network routing issue outside of our data center partner’s facility. Network Engineers have rerouted all traffic to bypass the affected routes. We apologize for an inconvenience that this might have caused and will continue to monitor in case of any further issues.
Initial Report: [7:30 PM EST] Our team is currently working to resolve performance issues that are affecting some of our customers. We are aware that some sites are not available as a result of upstream network issues at one of our Data Center providers.
We are working as quickly as possible to resolve this issue and will update here as soon as we know more. We definitely apologize for this inconvenience.