Business continuity planning: an industry obsession

The highly specialized communication requirements of the modern trading room present a number of challenges when developing business continuity plans. In this high pressure environment, where immediacy is required in order to ensure that clients remain productive, failure to have a workable business continuity plan can result in significant loss of profit and a damaging blow to reputation. While it’s impossible to prepare for every possible crisis, steps can be taken to ensure that infrastructure is resilient enough to withstand disruption, and also provide the flexibility needed to respond to changing events.

Originally called Disaster Recovery, and now commonly referred to as Business Continuity Planning (BCP), the task of developing adequate, workable plans to direct activities in the event of a crisis is not an easy one. This is particularly true when looking at the unique infrastructure present in the trading room. In this pressure-filled environment, not only is 100% up-time something to strive for, it’s demanded. For any firm, this is a massive commitment, as it mandates delivery of a cost-effective workable technology solution to meet the heavy demands of the trading environment.

New technologies: Changing the face of BCP
Business continuity plans have begun to capitalize on the growth in the capabilities and increased adoption of next generation technologies in trading rooms. Bringing this together is the use of Internet Protocol (IP), which is transforming the trading floor into a full continuity focused environment.

The key benefit of IP is that it not only delivers the functions and facilities required for business continuity planning, but can provide numerous benefits across the entire enterprise, thereby moving firms away from the typical scenario of diverting resources to separate BCP-only projects.

New challenges for firms
The expanded always-urgent view of BCP has realigned the focus of organizational resources. Given recent downsizing in the financial services industry, extreme stress has been placed upon already-overloaded, limited staff resources. Many firms are now also realizing that in order to properly manage business continuity planning; some level of dedicated resource is required at a senior level. Business participation is also crucial to the success and backing of any strategic plan. These resource requirements come at a significant cost at a time when budgets are heavily focused on efficiency and revenue generation.

Combined with increased interest from senior management, shareholders, regulators and others, there is greater pressure to deliver to meet ever heightening demands for business continuity and resiliency. The BCP process has a significant impact on resource allocation, from staffing and budgetary limitations and corporate focus to government and industry relations. As if this were not enough, newly forged cooperation with competing firms and organizations brings collaboration and compatibility issues that must be resolved in order to deliver the level of preparedness demanded by all interested parties. But lack of preparedness is not an option if continuity of trading operations is to be ensured. Examples abound of companies’ unwillingness to dedicate adequate resources (budget and staff) to building and managing resilient technology infrastructure. In every instance, it is the CEO who must answer the question of “why” this was
not done when asked by productive clients, the media, regulators, shareholders and the board of directors.

No longer is the question of BCP whether or when. But how?

Finding the Solution to Disaster Preparedness and Business Continuity

Preparation for the unknown is of critical importance to any organization in the financial services industry. This is particularly true in the trading room, where activity is not only extremely timesensitive, but also highly confidential in nature. Maintaining functionality, management of information and the continued operation of the business is critical to ensuring client retention and market stability. Preparation requires careful attention to building a resilient infrastructure that supports firm-wide and industry-wide business continuity planning. This BCP activity not only helps to mitigate risks associated with the unexpected, but also helps to assuage the everpresent minds of government regulators bent on passing legislation that mandates a particularly strict course of action.

Clearly, an effective business continuity strategy depends on a robust infrastructure that can cope with every imaginable disaster scenario that can be thrown at it. The question is, therefore, how to develop and manage the evolving complexities of the technology that supports BCP in a way that is resilient and continues to meet increasing technical demands as technologies evolve, and takes into account budgetary and employee resource limitations.

Classification of potential disasters in the trading room
As the process of business continuity planning has evolved, a series of classifications has begun to fall into place. This classification system can be used by firms to plan for different levels of events more easily and accurately to delivery an appropriate response. According to industry communication specialist IPC Information Systems, three levels of classification have been identified:

Level 1: Loss of access to the trading floor
All telecom systems (i.e., PBX, turrets, voice recording) are operational;
Examples include bomb threats, natural gas leaks, asbestos discovery, weather
conditions or public transit strikes.
Duration: Typically short-term, from several hours to a few days.
Level 2: Loss of trading floor infrastructure
Loss of one or more telecom/power systems
Often requires activation of redundant systems or spare equipment.
Duration: Typically longer term, one-week or longer.
Level 3: Permanent loss of building/business district
Complete and/or permanent loss of a building, district or central office facility.
Requires firms to relocate their trading operations to a backup site
Duration: Indefinite interruption ranging from months to years. Potentially permanent.

These levels truly clarify the potential for different levels of real-world crisis. They demonstrated that whether a failure is within a single trading desk, or involves the urban infrastructure of a major business center, it all comes down to continuity and preparedness.

Trading Room Requirements
It is worthwhile looking at what features need to be incorporated into the Trading Room infrastructure for a full business continuity plan.

A resilient trading system that can sustain one or more failures, while still allowing traders to continue working A global log-on to give traders access to any turret irrespective of geographical position A facility to enable traders to take their telephone number and full contact directory to any trading position in the office network with a connection to the LAN or WAN A streamlined management system that can manage multiple individual switches as a single enterprise system, thus being more adaptable to multiple ‘event’ scenarios

BCP Tools
A number of technology tools are available to deal with the levels of crises, allowing for proper response based upon the agreed business continuity plan. These tools focus on keeping traders in business using tools that provide the same or similar functionality as those to which they are accustomed. Some of the major tools available today include:

Carrier Redirection: In the event that the production facility is unavailable or even destroyed, traders will need to report to a different location to carry on the business of trading. They can only do this effectively with the availability of the hundreds or even thousands of Private Lines they depend on for communication with their customers and trading partners. To insure the availability of these critical circuits at the BCP locations, the carriers that provide these circuits will deploy digital facilities to these locations. In the event of a disaster, the firm need merely notify the carrier and they will re-direct these private Lines to the BCP locations. As implied by the name, the lines are redirected to the BCP location and disconnected from the production site. With Carrier Redirection, a given line can be connected to one site at a time, not both.

Internetworking: If the production site is still up and running, but some or all of the traders are unable to access the floor, internetworking of the systems can provide duplicate appearances of private lines without the involvement of the carriers. That is, a given line may appear on a turret at the production site and on a turret at the BCP site simultaneously. This would enable the traders who relocated to the BCP site to access lines immediately, without the need to wait for carrier re-direction. In addition, if only some of the traders re-locate, traders at the production site and the BCP site can still work productively, both having access to the same lines, with the lamping and signaling supervision required for orderly operation. Of course, internetworking requires that the turret switches at both locations are up and running.

Virtualization: Trading desks are large specialized pieces of furniture with room to house turrets along with their complementary market data screens. BCP sites therefore will usually be large enough to accommodate significant numbers of traders with their equipment. However, if space and/or resources are very tight, virtual, or soft turrets, may be utilized as a short term solution. This “virtual” solution will eliminate the need for backup trading floors to be built. Instead, traders displaced by a disaster can be relocated to existing space in normal office areas. All that is required for a soft turret is a PC and the soft turret application. Of course, management would need to identify non-critical functions whose personnel would be displaced for the duration.

BCP Architecture
The tools previously discussed can be combined to create a resilient BCP architecture to deal with virtually any type of disaster. The examples below illustrate the use of these tools. Ideas from each example may be employed to tailor a solution to a firm’s individual needs.

Site Mirroring: By far the most common BCP scheme, Site Mirroring involves the full or partial duplication of the production trading floor. The location of the BCP facility is generally chosen to be some distance away from the production floor.

Site Mirroring is used to provide a seat for some or all of the traders who may be displaced from their normal place of business when disaster strikes. Usually, to conserve resources only a portion of the traders will be accommodated at the BCP facility. The rationale as to which traders will be relocated may be dynamic over time. In any event, the list of traders with seats should be established before disaster strikes.

Site Mirroring requires enough backroom and desktop equipment to physically support the actual number of relocated traders. In addition, network facilities must be in place to interconnect the production and BCP sites. Provided that the production floor is still accessible and operational, this internetworking will allow traders at either floor to access their private lines. However, in the event that the production site is off line or even destroyed, some means of re-directing private lines to the BCP facility must also exist.

In order to transition smoothly from their normal desk to their assigned desk at the BCP facility, the trader must have his profile, i.e. his arrangement of lines and speaker channels, available to him when he arrives. Maintenance of these profiles and their portability requires database administration that is essential to the overall strategy.

Distributed BCP: For institutions that already have multiple trading floors, Distributed BCP may be employed to eliminate the need to build new floors and the backrooms to support them. Instead of building dedicated facilities for BCP, firms merely expand the capacity of existing systems to accommodate traders who need to relocate in the event of a disaster. This involves adding the additional shelves, power supplies and cards to the switching equipment to support the relocated traders, as well as the desks themselves. For the extremely resource constrained, the latter can be avoided if a “triage” plan is utilized, whereby traders at the various locations are displaced by relocating colleagues. As mentioned before, a list of who will sit where and who will be displaced must be determined beforehand and revised as needed on a dynamic basis. Of course, as in Site Mirroring, internetworking among the different locations and provisions for redirection of private lines must also be in place.

VoIP and the Thin Building: Traditionally, trading turrets are tied in a point-to-point configuration to backroom switches located in the same building. An optimal design for BCP can be achieved by separating the backroom switching equipment from the user interface. The “thin building” – “fat data center” architecture has been utilized for years in computer/data communications systems. Main frames and servers are kept in hardened data centers generally located outside of urban centers, while users and the PC’s or terminals they require are located where their customers are. With the advent of Voice over IP (VoIP), this model is now highly applicable to mission critical voice as well.

The main advantage of the thin building is its ability to separate from the fat data center. Only the equipment necessary for human interaction need be located at the actual trading location, generally the turrets themselves. The data centers may be built in areas with attributes that lend themselves more to hardened sites. Suburban or rural areas generally make it practical to build one or two story structures. Most security experts agree that these simpler structures are much easier to harden against disaster.


While the thin building architecture could be implemented using TDM technology, its true value is unleashed with VoIP. VoIP telephony involves an asynchronous connection between the end-user and the switch, thereby removing the dependency between the trader and a given physical location. The back office switch can now be located anywhere, well away from the trading floor.

Turrets with a native-IP interface are no longer tied to a specific location. They can be deployed away from the switch, and even to multiple geographic sites. Because IP networks carrying all types of traffic - voice, data and video - VoIP-native trading systems do not require a dedicated voice connection. In some configurations, they do not require local back office hardware. Main switches can be linked together through IP into a single enterprise wide system. This enables firms to manage bandwidth usage and provide BCP options in case one site should become inaccessible.

With VoIP, rapid reassignment of resources and automatic fail-over to surviving equipment is much easier to implement.

Today’s technology provides a wide variety of tools to mitigate the business effects of a disaster. The most effective solution for any financial institution will probably involve a combination of some or all of the above techniques. Management now has more options than ever before to create a BCP solution that is right for their institution.

The Right Technology Partner
There are many companies that can say that they provide high-quality technology services to trading rooms. There are many others that purport to deliver end-to-end solutions to the varied requirements of financial services firms. But it’s important to retain the services of a provider that fits with the requirements of the individual firm. No two trading firms are alike, just as how no two technology providers offer the same services in exactly the same way. There are a few things
that any firm should consider when selecting a service provider:

Proactive Attention: A technology provider should offer solutions to problems that the client does not yet know exist. This long-term, 360-degree view of development and delivery of trading desk technology helps to ensure evolution of functionality as business
requirements change to meet an ever-adapting market, and also helps to minimize development costs over the long haul.

Industry and Functional Touch –Points: A provider should always be a recognized leader in delivery of trading room technology. Anything less and you run the risk of misunderstanding of the unique challenges of trading room technology requirements. With a recognized leader, you gain the expertise and focus of a company committed to technical prowess.

Technology Focus, Financial Services Background: Many providers can offer a strong background in technology development. Industry focus is important to ensuring compatibility with related firms, partners and clients.

IPC: Obsessed with Service and Trust
IPC understands the necessity and importance for all financial institutions to implement best practices aimed at achieving the most reliable infrastructure and facilitating rapid recovery from any outage. As an industry leader for more than 30 years, IPC leverages the knowledge and experience gained by building more than 300 trading floors a year, to provide a full range of business continuity solutions that support the needs of thousands of firms and their traders.

IPC’s focus is on ensuring clients’ operational resiliency. Ever aspect of what IPC does delivers reliability to its clients–from design to manufacturing to implementation to support. This means that a firm’s information systems, people and business processes will always be protected and connected regardless of the type of disruptions.

IPC recognizes that there is no one ideal business continuity solution to fit all firms. While the impact of events on firms can vary in size and scale from temporary single component failure through to complete business district outages, there are a wide range of measures that can be employed to limit this impact. In most cases, trading firms will employ a number of business continuity measures in order to develop a robust and flexible system with adequate redundancy. IPC is able to provide expert advice to all of its clients to make sure they have the most effective business continuity solution

IPC has long pioneered the use of networking lines over TDM networks to enhance the availability of line resource in case of an incident. By adapting networking to work over the corporate IP network, a firm will be able to share these lines over greater distances and at lower cost. This offers a cost effective method over the traditional method of provisioning a separate trading room for business continuity.

As part of this development, the facility for any turret to be able to access any Alliance switch resource at either a main site, remote site or indeed any site within an organization will be provided. In terms of planning for business continuity this offers a completely new level of resiliency at any position. A Trading Room manager can effectively back up any set of Traders that he requires and use the whole set of resource of his organization. The ‘failover’ scenarios are planned in advance by system management and will eventually move to a fully dynamic model of failover by position. The ability for an IP station to interface to any switch in the infrastructure will become a key component of mitigating risk of any single failure in Trading Floor architecture. This allows a level of flexibility of BCP implementation not available to date, based on the firms’ exact requirements.

If there is a need to move to another site then the facility to switch over a Traders identity, all of his connections to counterparties and the ‘button sheets’ will be made available. This delivers the often sought ‘Global Free seating’ capability where the Trader is able to access his position from whatever central resource is available.

Additionally in moving to the wide area a facility to provide access to IP based private lines (Enterprise Voice Services) simultaneously on multiple floors will be delivered. Perhaps most importantly, this communication interface is present regardless of a loss of infrastructure at any one of the trading floors.

Finding the Right Solution
What makes a solution right for each firm is dependent upon their unique requirements. The common thread for most organizations has become the use of next generation technologies to simplify and enhance the efficiency of communications within and between trading desks.

At the end of the day, the right solution is going to be the one that delivers such that the technology is transparent to the end-user, offering a simple tool that helps them to communicate with, and provide service to their clients. To the end-user, down-time is not an option. To the end-user’s manager, down-time is lost efficiency and leads to client complaints. To the risk or telecommunications manager, down-time means lost sleep. To the CIO, down-time means having to write a lengthy, detailed explanation as to the cause and resolution of the issues for the COO and CEO. To the CEO, down-time means media attention and lost revenue. In other words, down-time is not an issue that can be hidden under the corporate rug. It’s a serious problem that needs to be considered in advance.

The industry’s obsession with Business Continuity Planning can be simplified by providing a stable, resilient technology infrastructure to trading rooms. Selecting a respected provider such as IPC to deliver the specialized technology requirements of the trading room ensures maximum efficiency of resources and ultimate support to business continuity planning.