Shifts in the Network During COVID-19 Pandemic

Francis is the Director of Network Operations and is responsible for technology and service management functions including; the Network Operations Center, planning and engineering for the technology and networks, which is at the heart of the company’s residential, business and wholesale services in Hawaii. Francis started his career in telecommunication in 1991 with GTE Hawaiian Telcom. Prior to joining the company in 2016, he spent 18 years on the mainland with Verizon and has held leadership positions in several areas including OSP Engineering, Operations Support, Global Network Engineering and Field Operations. We sat down with him to ask how Hawaiian Telcom has prepared for the COVID-19 pandemic and how the network is holding up to the increased demand.

Network Traffic

Q: Since the COVID-19 pandemic started what kind of traffic are you seeing to the network and how does that compare to what is typical?

A: We have seen the traffic steadily increase since even before the Stay at Home Orders. Overall we are ranging about 30%-40% additional aggregate traffic across our network.

It has been a noticeable bump, but our network has been able to handle it. We have also seen a noticeable shift in the pattern of the traffic throughout the day. You now have parents working over VPN, kids doing distance learning, along with streaming, gaming, etc., all of which is increasing traffic from the home. While traffic has increased during the usual peak evening hours, it has increased quite a bit more during the daytime hours.

The capacity across our core network (where data traffic leaves your house and works back through network) is in excellent shape. Thanks to the hard work of our planners and engineers, we have the capability to handle increases to critical data and voice traffic as utilization grows.

"Thanks to the hard work of our planners and engineers, we have the capability to handle increases to critical data and voice traffic as utilization grows."

Network Preparation

Q: How are you preparing for additional volume spikes from Hawaii’s stay at home order as well as unforeseen circumstances such as inclement weather.

A: Voice and data network engineers in our Network Operations Center (NOC) have been closely watching the network and capacity as the situation around us continues to change. Where we are seeing traffic increase or patterns shift, we are able to make quick adjustments to augment capacity and update the networks. We are watching not only the short-term shifts in the network, but also planning ahead for long-term shifts in usage, such as the long-lasting onset of telehealth capabilities, or the likelihood that some businesses may increase their work from home capabilities on a more permanent basis.

We have found ourselves in good shape to handle the current Work from Home Orders, but we are also in hurricane season, which adds a chance for issues across the network due to severe weather at a time when people are working and learning from home. There are things that are out of our control, like power to the customer, but where we have cable cuts or other storm related issues, our central office and field operations teams are ready to act quickly to restore service.

Critical Infrastructure

Q: How are we helping critical local organizations stay connected during this pandemic?

A: With the Work from Home Orders came an increase in escalations for extra capacity and speed for businesses. Schools, small businesses and hospitals have been quickly increasing their capacity to provide services in new ways. As usual, our network, field and central office teams have really stepped up in assisting wherever possible. These types of scenarios really give our employees a chance to shine, as they are extremely customer focused and quick to respond when needed.

Beyond additional requests, we have a responsibility to plan and be on top of as many scenarios as possible to help keep our customers up and running. In certain areas, this has been imperative. For example, we manage the delivery of local 911 calls and handle the voice and data communications for local governments, hospitals and health services. We recognize the importance of maintaining a constant flow of communications, so our operations teams continue to step up in these unusual times.

Preparing and Managing through COVID-19

Q: How have you prepared and managed network operations during this COVID-19 pandemic?

A: Hawaiian Telcom has a responsibility to keep our employees safe, but also to keep the network and critical infrastructure up and running no matter the situation. We have business continuity plans that have been tested and are now being put in place. However, as you can imagine, there is no good way to plan for all of the situations we are currently seeing, and no playbook to pull from. It's really about building an infrastructure for fast decision making to respond and act as quickly as possible. Over-communication, clear decision-making and immediate responsiveness are important aspects of our current business continuity team.

"We have business continuity plans that have been tested and are now being put in place."

Moving our Network Operations Center (NOC) employees home is just one example of the complexity that we are facing with this unprecedented situation. Our previous Pandemic Business Continuity plan calls for us to separate the teams between a primary and secondary location in order to mitigate exposure risk. We have tested for this and had even put in place a concept called “air gapping” before many businesses were even thinking about working from home. But sending our NOC employees home had previously been something we didn’t believe was technically possible. Once again, the team stepped up to ensure we could offer any employee who preferred to work from home the opportunity to do so safely and effectively. It speaks to the dedication our team has to our employees and keeping them safe.

Network Connection Troubleshooting

Q: What can customers do to troubleshoot issues they are seeing with connectivity? How do they determine whether it is an application, or their internet connection causing the issue?

A: The work and learning from home environment can be challenging, especially as new tools such as Zoom become more popular in the market (and overwhelmed with new users and traffic). For users having issues, I recommend starting by going to speedtest.net to test your internet speed and connectivity. Try other applications and if it’s the case where everything is slow, then follow your typical troubleshooting: rebooting a gateway, call technical support, etc. Another good place to start is downdetector.com where you can see if the application you are using is getting reports of issues across the community.

With added users and volume in the home, WiFi connectivity and bandwidth may also be an issue. If you don’t already have Hawaiian Telcom’s Premier WiFi solution, I would recommend it. As a best practice, run a hardwire connection from your device to the gateway and take away the wireless connection altogether. It’s not always technically possible, but if you are the working parent in a house full of online kids, this will help to fully take advantage of the high speed internet coming into your home.

Network Redundancy

Q: What kind of network redundancy do we have in place?

A: With internet traffic lighting up all over the country, one of our biggest concerns has been our connectivity to the broader Internet so that all of your cloud applications, games, and streaming continue to function as expected. To ensure we have redundancy, we connect to the Internet through multiple points, including locations in California and Oregon. We have multiple paths to connect to those bigger providers so if one link goes down the other ones pick up the slack without impact.

Tips for Working From Home

Q: Are there any special considerations for people working from home?

A: As an employee now working out of the home, we must take steps to make the house look like an office. Buying a monitor, connecting directly to the gateway, etc. There are a lot of employees who have access to data that should not be shared with others, so security should be considered when finding your perfect spot so setup. Make sure sensitive customer data is not available to all walking through the kitchen or close down applications that you aren’t using when you are done. To be sensitive to other home workers, consider logging out of VPN during lunch or breaks. And of course, make sure others in the home know when you are on a video call so you don’t have unexpected participants joining in!

Our Commitment

Q: What would you tell people who have concerns about Hawaiian Telcom and their Internet service during this time?

A: Right now it is critically important that we as a company:

Keep our employees safe and healthy.

Keep our customers up and running. We are doing everything we can to keep our services running the way our customers expect and need. We know how important these services are, because they’re important to us as well. We won’t always be perfect, but we hope people understand that no matter how bad things get, we have people constantly focused on services and our customers because that is what we do. We are Hawaii’s technology leader, it’s our friends and Ohana using our services. There was phrase used when I first started my career at Hawaiian Tel, “Beyond the Call” and I see it every day. We have the most dedicated employees in the industry, and they consistently go above and beyond to keep their Ohana and friends working and connected to one another.

"We have the most dedicated employees in the industry, and they consistently go above and beyond to keep their Ohana and friends working and connected to one another."