In our previous blog posts, we’ve covered the core components, orchestration capabilities and automated analysis and ChatOps features in our in-house Security Orchestration, Automation & Response (SOAR) application. In this final blog post, we will briefly cover one of the system’s automated response capabilities that allow it to take action and respond to alerts without human intervention, and close out the blog series with two important lessons learned from implementing SOAR at Appian.
Automating security to “get out of the way.”
Like many organizations, Appian uses technology on company-managed devices to prevent access to potentially malicious and work-inappropriate domains. This is a good way to reduce the attack surface of an organization, as phishing emails linking to lookalike or suspicious domains are one of the most common security threats associated with credential theft and malware, including initial access vectors for dreaded ransomware. We also monitor for new potential spoofing domains that may be used to impersonate Appian web properties. For most automation-minded security professionals, the first thought that comes to mind when hearing this is to automatically quarantine these potentially malicious domains when they are detected, pending investigation and a final decision by a security analyst. This is a reasonable action, and the SOAR application can do that automatically. However, to illustrate how automation can be harnessed to better serve the customers of the Appian security organization and eliminate false positives, we’d like to provide a more interesting response example: how we automated removing a domain filter.
False positives in domain filtering can be annoying for users. Sometimes they represent a small frustration, like being unable to load some blog post from a little-known site. In other cases, they can significantly cripple productivity, as happened to an Appian consultant. A customer abroad had recently signed up for Appian, launched a new development instance on a new domain, and hired the Appian Customer Success team to build an application. As the customer’s business day started, it was the middle of the night in the US, and the consultant was faced with a screen telling them that their freshly launched development instance was blocked. They submitted an unblock request, but at the time, those were going to a separate email inbox that was only monitored by the Appian security team in the US during business hours, and it took many hours for the issue to be addressed. Needless to say, this significantly hampered the consultant’s ability to work that day and represents a prime example of security getting in the way of productivity.
This story bothered us, and we decided to make sure that it would not happen again. How we solved this issue is a good illustration of the entire feature set of the SOAR application:
- Our existing event ingestion capabilities can be used to ingest email based alerts, so we started by configuring the emails to be sent to Appian, and triggering a process that parses the email into structured data and stores the event in the database.
- Alert in hand, we apply our standard automated threat intelligence enrichment to the domain name itself and other indicators parsed from the request. This includes sandboxing the domain to get a rich data set of all HTTP requests/responses encountered when accessing the domain.
- Each fully enriched alert goes through the Rule Engine for evaluation against a predefined set of rules that tell the system what, if anything, to do with it.
- We configured API integrations with the domain filtering system to automate blocking and unblocking domains, and added them as response options for both the Rule Engine and manual analyst actions from the context of an event.
- For additional analyst efficiency and user benefit, the system emails users who have requested a site to be unblocked when an unblock is done and marks the event(s) as resolved pending final review.
With this workflow in place, we were able to work with our product team to understand the HTTP request/response flow, cookies, and other HTTP activity associated with accessing a legitimate Appian instance. For operational security reasons, we can’t disclose the exact things we examine, but the end result is that the Rule Engine can use a set of checks to evaluate whether the requested domain is an Appian instance, automatically unblock it, and send the user an email. What might in the past have ruined a workday or at least been a waste of several hours is now automatically resolvable in minutes, 24/7.
Lessons learned from automating security.
Having covered all of the four letters in the SOAR acronym in describing our application, we wanted to conclude this series by summarizing two of the most important lessons we learned from introducing automation into our daily security operations. While these lessons are derived from a security context, we think they are more widely applicable to any low-code automation project you can take on with Appian.
1. Automation should be customer-centric.
Security teams considering a SOAR implementation typically start by thinking about what it can do to improve security outcomes or reduce analyst workload. These are worthwhile goals in an era where security breaches are becoming increasingly common and costly, but analysts are only one customer of security automation. All other teams are the customers of the information security department, and security cannot come at the expense of users being able to perform their best. Much like how a poor customer experience with a company makes you less likely to remain their customer, any instance of security efforts hindering a person’s ability to successfully do their job undermines their relationship with the information security department. These are important relationships to preserve, as they are crucial to the success of any information security program. Verizon’s 2021 Data Breach Investigations Report found social engineering to be the most common attack vector in breaches, and 85% of breaches involved a human element while only 3% involved exploiting software vulnerabilities. Since, in addition to being the first line of defense against social engineering, people are responsible for 100% of great things that businesses achieve, they deserve to perform their best and have a positive perception of information security so they are empowered to ask for help when they need it. Further, technically skilled users technically skilled users who feel impaired by security are masterful in circumventing security controls – just ask any high school IT teacher who has tried blocking video games in class.
Automation can help. Here, we discussed how we have used it to improve response time for legitimate requests. In prior posts, we explained building a chatbot to make information more easily accessible to engineers and administrators in real time, and using data to identify false positives before anyone needs to field questions about them. As much as these features are in service of analysts and designed to benefit their efficiency and work-life balance, these initiatives also help us provide a better experience to the people who maintain and develop our business while promoting good security practices. The benefits to us are obvious – when we improve relationships with other departments and avoid security being a bottleneck, we can reach out and count on help from others when we do have genuine causes for concern.
2. Simple improvements can have a powerful impact.
At Appian, we are big believers in impact. We offer the Appian Guarantee to help our customers quickly deliver impact from their first application. Our CEO Matt Calkins likes to emphasize working to impact, not completion. The completion of a SOAR implementation is itself a fuzzy concept, as there will always be a new security alert type, threat intelligence source or automation playbook that can be added. It’s easy to be tempted to start with building elaborate playbooks that address a very specific issue or alert. However, in retrospect, the most impactful things SOAR brings to Appian’s security operations are three relatively straightforward improvements to the way we work:
- Centralizing alerts into an application with case management means there is one place for viewing security events, dramatically improving visibility and reportability and eliminating time wasted jumping between tools or using inefficient methods like an email inbox to manage work.
- Automated threat intelligence collection means there is little to no need for analysts to manually query intelligence sources, saving 10-60 minutes for every alert out of the tens of thousands reviewed annually.
- Rule-based automation for dismissing known false positives means over half of alerts don’t need to be manually reviewed at all, and investigation steps performed once can be translated into rules that continually compound time savings with every repeated alert matching a rule.
These core features are the basic foundation for more targeted automation, and are likely going to account for the vast majority of time that SOAR saves for analysts in the long run. We can build on them to add specific responses to alerts where response time is of the essence, and continue to further strengthen our security posture. Getting the foundation of a few things right can be tremendously impactful over the course of time.
If you are a current or aspiring InfoSec professional looking for a new place to make an impact, be sure to check out our open positions! Appian Information Security is currently hiring across all teams. In Security Operations, we have openings from a new grad Information Security Analyst all the way to an experienced SOC Manager. Likewise, our Security Engineering team has opportunities for InfoSec engineers ranging between a college hire and an experienced Principal Security Engineer.
