Server update - 9th October


by Team Appkettle October 09, 2017



New User Summary
(*Updated 9th October)

From September, a number of UK and EU companies (including Appkettle) have been affected by a multi-server supplier with Amazon Server issues, more information from our original blog post can be found here and the follow up here.

(*)The server supplier has caused significant inconvenience to Appkettle and to all Appkettle users. We deeply care about all our users and hope all services have been successfully restored by our supplier within the next week.
  

What server symptoms to expect?
(*Updated 9th October)   

In September, the supplier's server encountered an issue with intermittent slow connection speeds (latency) and a symptom around 5 pm (UK) where all service connections dropped offline, including Alexa, IFTTT, and 3G/4G services. 

(*)After a late review on Sunday 8th October it was agreed by the supplier, external server specialists, and other companies, to take the supplier server temporarily offline (3G/4G, Alexa, and IFTTT) from 5 pm yesterday.

All engineers have now worked through the night and today to get the server with upgrades back online initially by 5 am UK time on the 10th October - tomorrow.

Once this initial phase is complete, a full resolution is planned iteratively over the next week.
Please read on below for the plan and further information.

Please note - WiFi app functionality is unaffected throughout.
  

What progress has been made?
(*Updated 9th October) 
 
  • In the past two weeks, supplier and Amazon engineers have analysed, traced and debugged their servers. 
  • A backup (clone) server was set up to minimise the event of a primary server failure. However, this backup did not trigger when the issues re-occurred.
  • The server configuration, bandwidth, and CPU usage are more than sufficient for all service requirements.
  • The main issue found was the number of concurrent connections and the load balancing on the servers between device types.
  • (*)On Sunday 8th October, server specialists have begun working overnight on the server and have made significant changes as defined in the 'what's next' section below.

  

So what's next?
(*Updated 9th October)

 

*Yesterday at 5 pm (UK time) it was decided that the server is taken offline temporarily so critical work could be completed more effectively over a longer duration to finally resolve all issues. 

*The server specialists, Amazon, and the supplier engineers have devised the following resolution plan for their servers:

1) Separating device types across servers
2) Develop load balancing across all servers
3) Observation of each device status on each server and log this performance information.
4) Improve and refine the server system based on the logged information.

*The server specialists and engineers will complete these initial changes by 5 am UK time, these will be refined and improved over the next week to a complete solution for all users.
  

Our thoughts.
(*Updated 9th October)

 

*We have been doing everything possible and actively living the inconvenience of this supply issue outside of our control to all Appkettle users for over 4 weeks. We can empathise and understand how it is affecting product use and enjoyment and that we can only apologise for this supplier interference.

*We have escalated this issue to the best of our ability, working 7 days and up to 20 hours a day to ensure we support, coordinate, push and get a resolution deployed safely, efficiently and effectively within the shortest possible timeframe for all Appkettle users.

*At present we cannot switch providers as suggested by some Appkettle users, as this could prove more problematic when transitioning data, systems, and functionality (plus it could take longer). However, after this week we will certainly review taking further actions as necessary against the supplier and to have a full solution available.

*We can confirm that our supplier has assured us that this issue will be dealt fully within the next week to the best of their abilities and all engineering both internal and external is actively working on it.

 

  

Thank you for your patience and support

 

We would like to apologise for supplier issues affecting your product enjoyment and experience currently.

I hope you can understand this is a temporary situation outside of our control and we are working 7 days a week to assist wherever possible to resolve the issue to all Appkettle users very soon. 

 

 

 

 




Also in Blog

Server Update - 6th October
Server Update - 6th October

by Team Appkettle October 06, 2017

Read More

Server Supplier - Setting the Record Straight
Server Supplier - Setting the Record Straight

by Team Appkettle September 20, 2017

Read More

Our Favourite Applets - TOP 5
Our Favourite Applets - TOP 5

by Team Appkettle September 08, 2017

Read More