Service Interruption Update

We wanted to give you an update on the service interruption issue that has affected 1/3 of our Cub Scout customers. This issue does not affect Boy Scout customers.

As you may know from a past Announcement, on August 22nd one of our Database Servers experienced a hardware failure. This server is the home of approximately 1/3 of our Cub Scout customers. The faulty hardware was replaced and the server rebuilt. No data was lost.

Since then the Database Server has experienced periods of connectivity loss, during which customers cannot access their SOAR website. There is no data loss due to this issue.

In an effort to minimize downtime for customers while still allowing us the opportunity to diagnose what is going on, a monitor was installed to test connectivity every 3 minutes. This means that the maximum downtime for customers is 3 minutes. It also give us enough time to gather all the logs and current state of the service to further diagnose what is going on.

During the 3 minute period we have put a more user friendly page in place than the “Unable to connect to database” screen that was previously displayed. This will assist in removing confusion and worry for your members and also set expectations that they can again access the site in a short period of time.

We have ruled out hardware as the issue. On September 3rd the Database Server was moved to a brand new server. The same issue persists.

We have narrowed down the issue to something within the database itself, that when accessed is causing the Database Server to cycle and lose connectivity. All of the database checks come out fine of course ! The logs that we have gathered are providing us a better picture of commonalities at the time of the issue. We hope to have this issue resolved before the end of September.

We know that September is a busy month for scouting. We believe the “3 minute” approach described above, while taking longer to resolve the issue, is much better than the alternative - taking the server down for an extended period of time during which access to your SOAR website would not be available.

We will continue to update you on progress towards resolving this issue.