[resolved][2021-03-10] ECO platform is down: ThingPark Community Portal, POC, Dev1 total outage
2021-03-27T22:00:00Z
- 🛠 community.thingpark.org:
- ✔ subscriber creation is working again, but still not available for customers as Interop Engine is not working yet,
- ✔ Single Sign-On (SSO) to interop engine is also working
- 🛠 Interop engine validation needed.
- ✔ TPXLE + ADA working for
dev-opeinstance- 🛠 TPXLE not working for community instance without portal
- ✔ All needed operator instances are now created.
- 🚥 Next in line:
- Subscribers, basestations and devices migration.
- TPX decoders and drivers
- Restore Partners sites
Please refer to the status page:
https://status.thingpark.com/
---
Incident history:
Thingpark Community, PoCs and Dev1
❌Network Server for PoCs
Major incident
❌Network Server for Dev1
Major incident
❌OSS API and GUI
Major incident
❌Thingpark X Location Engine
Major incident
❌ThingPark X IoT-Flow
Major incident
❌documentation platform Major incident
- 2021-03-10T09:35:00Z: the hosting company (OVH) just communicated they won't be able to restore their datacenter today.
2021-03-10T06:51:00Z (morning, Paris time) : 🔥 A datacenter hosting some of our services has burnt down 🔥
2021-03-11T13:50:00Z
after the fire 🔥 that damaged multiple datacentres in Strasbourg, France, yesterday, the entire ECO environment is still down.
OVH is in disaster-recovery mode. In the next few days, they will let us know what services / data are gone and what we can restore. We will then assess the situation.
Our teams are working on restoring our own manual backups to a different location.
after the fire 🔥 that damaged multiple datacentres in Strasbourg, France, yesterday, the entire ECO environment is still down.
OVH is in disaster-recovery mode. In the next few days, they will let us know what services / data are gone and what we can restore. We will then assess the situation.
Our teams are working on restoring our own manual backups to a different location.
2021-03-18, morning, Paris time. Disaster recovery plan is ongoing.
- ✔ The new
community platform is up, including
DX-API and
IPSec.
- 🛠 IoTFlow, ThingPark X Location Engine, Interoperability Engine are not back online, yet.
- 🚥 next in line for re-establishment:
- TPX IoTFlow
- ThingPark X Location Engine (TPXLE) + Abeeway Device Analyzer (ADA)
- migration of operator instances, subscribers, basestations (gateways), devices
- provisioning of Community offers (including Interop)
- 🛠 IoTFlow, ThingPark X Location Engine, Interoperability Engine are not back online, yet.
- 🚥 next in line for re-establishment:
- TPX IoTFlow
- ThingPark X Location Engine (TPXLE) + Abeeway Device Analyzer (ADA)
- migration of operator instances, subscribers, basestations (gateways), devices
- provisioning of Community offers (including Interop)
2021-03-19, morning, Paris time. Disaster recovery plan is ongoing.
- ✔ IoTFlow is installed
- ✔ DX-API was updated to facilitate reprovisioning of subscribers and end-user accounts
- 🛠 ThingPark X Location Engine, Interoperability Engine are not back online, yet.
-🚥next in line:
- validate TPXLE and ADA
- achieve creation of all operator instances
- migrate subscribers, basestations (gateways), devices
- provision Community and Partner portals offers (including Interop)
- ✔ IoTFlow is installed
- ✔ DX-API was updated to facilitate reprovisioning of subscribers and end-user accounts
- 🛠 ThingPark X Location Engine, Interoperability Engine are not back online, yet.
-🚥next in line:
- validate TPXLE and ADA
- achieve creation of all operator instances
- migrate subscribers, basestations (gateways), devices
- provision Community and Partner portals offers (including Interop)