Improving Desktop Security

By Lance Perry, Vice President, Cisco Information Technology Group, IT

For a global enterprise, protecting desktops from constantly changing malware and other threats is a never-ending job that can consume far too much of the IT department's time and budget. It's not simply the effort of warding off threats or cleaning up after an intrusion; threat-protection solutions have can be time-consuming to deploy and manage. There is also the difficulty of making sure that your security methods don't get in the way of doing business. They must be flexible enough to enforce different policies for different connection methods, departments, or users, and unobtrusive enough to not drive users to attempt to circumvent them.

Effective security solutions can be costly, but the cost of failure can be even higher. For instance, in 2003, when the main issues were simply viruses, worms, and trojans, Cisco spent an estimated US$250,000 on IT remediation and cleanup efforts for minor malware outbreaks, and severe incidents cost up to US$2.5 million. And now, in 2008, IT has to mitigate all those threats plus spyware, rootkits, botnets, targeted attacks, and rising threats from social networking sites. In another two or three years, who knows? One thing is for certain: it won't get any safer out there. The experience of Cisco IT with desktop and server protection illustrates both the challenges and the benefits of getting desktop protection right.

70,000 Targets to Protect

Cisco IT is responsible for protecting more than 70,000 desktops worldwide. Although IT had deployed leading antivirus solutions, these were not enough, both because of the speed with which new infections spread and because they could not mitigate zero-day threats whose signatures were unknown. "Traditional antivirus solutions are at best 90 percent effective, but 10 percent of malware still gets through," says John Stewart, vice president and chief security officer at Cisco. At one point, Cisco IT spent US$1 million quarterly in operational costs related to cleanup efforts. The spending level needed to battle escalating attacks wasn't sustainable.

Cisco IT also needed time to install vendor patches for newly discovered operating system and application vulnerabilities. "We prefer to perform full quality-assurance testing on all patches before deploying them in the Cisco development environment," says Paul Mauvais, a senior security architect at Cisco. "In the past, we had to balance the need for thorough testing with the need to quickly install patches to avoid exposure." This is especially difficult in a global corporation operating across multiple time zones, and where employees often work off the Cisco network for days or weeks at customer sites, leaving them vulnerable even after patches have been distributed. The company needed a new solution.

Looking For Misbehavior

In 2004, in collaboration with the Cisco Information Security (InfoSec) Services team, Cisco IT adopted Cisco Security Agent, which complements antivirus solutions and patching efforts by looking for unusual application behavior, such as an attempt to access email, change a registry key, or connect to an unexpected network. IT can specify how Cisco Security Agent responds to such an event: allow the action, deny it without informing the user, or give the user the choice of how to respond.

Because its detection capabilities are based on behavior rather than a signature, Cisco Security Agent can detect zero-day attacks. And, unlike personal firewalls, even if the detected event originates from a trusted source, Cisco Security Agent can stop it, thus protecting desktops against threats that originate within the Cisco network. Cisco IT used some of the predefined policies supplied with Cisco Security Agent along with custom policies that allow activities like automated software delivery, launching the WebEx conferencing applet, and so forth.

Starting Small

These policies were developed during a 400-user pilot deployment using a Windows server at Cisco headquarters in San Jose, Calif. The pilot was run by a project management team that included representatives from the Personal Computing Solutions group, which manages PC support; engineering; InfoSec; and the helpdesk support team of the Global Technical Resource Center. Global employees who volunteered to participate in the pilot agreed to provide specific types of information at specified times and were instructed to download the Cisco Security Agent software from an internal Web page.

The pilot enabled Cisco IT both to develop policies that don't require too many actions from employees, which tend to hamper productivity and encourage circumvention attempts, and to profile the expected behavior of complex applications not covered by Cisco Security Agent's default settings, including the one used to distribute software to desktops. "We did not have to profile all 10,000 applications in use at Cisco," says Mauvais. "Profiling a few of the more complex applications gave us a starting point that we could refine after testing within the pilot environment."

Pilot participants were asked to operate the application as they normally do over the course of a day, which gave Cisco IT the information that it needed to identify allowed actions and set up the policy to deny all other actions. Cisco Security Agent was configured to send all messages to a central management console to avoid disrupting employee work and give IT the ability to learn about expected behaviors. It also permitted Cisco IT to adapt its global support processes by defining criteria for escalation, and gave the team confidence that deployment would not overwhelm support or impede employee productivity.

Rolling It Out

From that point, deployment proceeded swiftly using Cisco's software distribution application over the Cisco Application Content and Networking System (ACNS) distribution network. It took less than three weeks to extend the protection to over 70,000 desktops.

But deployment did not stop there. For instance, Cisco's Service Provider Video Technology Group now uses Cisco Security Agent in its manufacturing facility in Juarez, Mexico. Engineers there need to install and use a wide variety of applications on test stations, which creates a fertile breeding ground for virus infections. It could take upwards of a month to identify and cleanse all infected stations in the wake of an attack.

By deploying Cisco Security Agent on all stations with strict policies governing which test stations can communicate with which servers, and preventing test stations from communicating with each other, Juarez IT group effectively segmented the network in a way that limits or eliminates the spread of viruses. It took the group about two months to gather the information necessary to develop these policies. "First we analyzed 100 applications to determine typical transactions, processes, protocols, ports, and IP addresses," says Galo Guzman, manager for Juarez IT security. "Based on that information, we configured several dozen rules for allowed application behavior on test stations." For example, some desktop applications are set up default to call home to an Internet server to update themselves, and the IT group decided to disallow this activity.

The benefits were immediate and measurable, with no impact on the manufacturing process. "Infections have decreased significantly," says Galo. "In addition, the Cisco IPS [Intrusion Prevention System] is detecting far less malicious activity on manufacturing network segments." The number of incidents reported per month decreased from 31 to 11 the month after Cisco Security Agent was deployed, and only one trouble ticket was issued in December and January of 2008.

Major Headaches Removed

Overall, the deployment of Cisco Security Agent has radically improved desktop security. "Cisco used to regard desktops as one of the most insecure aspects of the Cisco network," says John Ireland, system administrator for Cisco's host security architecture. "Now we regard them as one of the most secure."

The Solution proved its worth during a major virus outbreak just two months after deployment. Although Cisco IT applied the new virus filters to Cisco's e-mail servers just minutes after the virus hit, e-mail activity had already spread the virus throughout the company. But of all the desktops running Cisco Security Agent, only a small fraction became infected. On those desktops, employees had clicked "Yes" twice when warned that a suspicious application was attempting to write to the run key of their registry and access e-mail resources, so Cisco IT has since changed its policies to rely less on employees' judgment regarding whether or not to allow suspicious application behavior.

IT is spending a lot less time and money dealing with desktop security issues, as well. Before deployment, it took 12 Cisco IT engineers to manage desktop security. Now, it only takes 20 to 30 hours a week to protect 70,000 desktops. The early notification of abnormal activity furnished by Cisco Security Agent enables early preventative action, resulting in a remediation cost savings of more than US$4 million annually in personnel costs for reacting to malware incidents. Furthermore, the behavior-based detection of zero-day events means that Cisco Security Agent catches a lot of malicious behavior right out of the box.

Finally, Cisco IT can now take its time testing vendor operating system and application patches, which has reduced application issues arising from new patches, reduced downtime from installation errors, and delivered better change management, reducing risk. "Uptime is critical in a manufacturing environment, so we welcome any solution that reduces the need to shut down and restart PCs," says Bob Scalise, director of enterprise security, Cisco Service Provider Video Technology Group. "Cisco Security Agent allows us to expand our patching windows to accommodate additional testing and validation, instead of being forced into a strict monthly patching schedule."