A lot has been written about the potential for privilege escalation in AWS, due to the use of permissive IAM permissions. These permissions, which may look benign on the face of it, can be exploited by hackers to grant themselves higher abilities within an AWS account.
Palo Alto Networks’ Unit 42 wrote about it. Rhino labs wrote about it. Tools like Cloudsplaining and Parliament were even created to find it. The challenge, now, is to catch it before the problematic IAM policies ever make it into the cloud environment.
You see, developers will often go and write IAM policies that will enable them to run their application. Oftentimes they will choose to go with more permissive policies, to avoid going through the painstaking effort of reducing the policies to the absolute minimum. It’s not coming from laziness, far from it. They have a feature to deliver, or a business service that needs to go to production. They don’t have the time to deal with this. Figuring out what specific permissions are needed is a whole journey of its own, as it’s not always obvious.
What’s more, once they push the code to production, no one wants to change the IAM policy, concerned that it will break a running application.
Why is this a problem? Why should you care about privilege escalation?
Let’s take an example – you have a Lambda function, running under a specific role. That function is processing input provided from the outside (say, users in the Internet) and is running in Python. Furthermore, that function deserializes its input using jsonpickle.
jsonpickle library actually has a known vulnerability, which can let attackers run shell commands through it. So, what happens once they can run a shell is important.
If the Lambda’s role is scoped down only to the permissions that are needed by it, the odds of the hacker getting very far are quite diminished. However, if the Lambda function has, say, the ability to trigger other Lambda functions, and even replace their code, you have a problem.
You see, a hacker may find the other functions, and begin replacing the code for each of them to something of their choosing. Most probably, something that will “phone home” (to their command-and-control server) and report what permissions it has. That way, they can iterate over each function until they find one that has the permissions they need.
This sounds like a lot of work, but motivated hackers are patient. They will look for the weakest points of entry, and work to gain more and more privileges as they progress, until they get to the pot of gold they’re looking for.
How do I avoid setting permission policies that open the door to privilege escalation?
You need to look for the potential privilege escalation when the application is being built, within the CI pipeline. Essentially, take every single permission policy you are creating, and pass it through a tool that will inspect it and tell you if there’s a potential for privilege escalation. Here are a few options:
Cloudsplaining is an open source tool, an AWS IAM Security Assessment tool, that identifies violations of least privilege and generates a risk-prioritized HTML report. It can also generate text-based analysis for policies provided to it on a one-by-one basis. Recently, it’s also been integrated into checkov, an open source IaC security tool provided by Bridgecrew.
Indeni Cloudrail will parse your Terraform plan, merge it with your cloud account (in memory only), and then analyze the IAM permissions each entity has. It will then use that to determine overly permissive policies that can lead to privilege escalation.
It will look for the known combinations of bad policies and highlight them, such as:
- Giving a role the permissions to escalate permissions (see example here)
Generally, giving a role any
iam:*permissions is risky. However, if you must do it, scope it – make sure the action can only be done on specific resources (like a group, or a role) where it makes sense. By doing that, you will be reducing the potential mobility of a hacker.
- Allowing someone to create a Lambda function, pass a role to it and invoke it (see example here)
Provisioning Lambda functions should generally be done only by a dedicated entity which is heavily guarded, whose usage is logged and audited. This is because the simple ability to run code under any role (“pass a role to it”) means they can essentially do whatever they want in your account. You should treat such an IAM entity as the equivalent of one with AdministratorAccess.
- Allowing someone to create access keys for other users (see example here)
Sometimes, you would want to allow an IAM user to create their own access keys. That’s fine. Just make sure the policy only allows the IAM user to create their own access key, and not a key for other users. This is achieved by setting the Resource component of the policy correctly.
(click on the links to see the Terraform code and how Cloudrail alerts about these violations)
When looking for such issues, it’s also important to ignore non-issues. For example, a group which has
iam:create* for itself, is not an issue (see example here).
Parliament is a great tool for doing this. It works best when policies are separate from the infrastructure code used to provision them. You can pass your policies through Parliament within the CI pipeline, and break the pipeline if Parliament finds an issue. If it does, the exit code will be non-zero, and you can stop the pipeline and have the developer fix the problem.
Many of the privilege escalation issues found by Cloudrail can also be found by Parliament. You can see how it works in this file in the GitHub repository. The examples given above for Cloudrail also apply in Parliament. Also, Parliament is an open source tool and was created before Cloudrail was released. It’s really a matter of how best you want to integrate this into your pipeline.
BTW, if you use Terraform, there’s a way to integrate Parliament with it easily, check out tf-parliament.
Policy Sentry actually helps you build good policies to begin with. It’s hard to figure out how to craft the ideal policy, with all of the supported actions out there. Policy Sentry allows you to build certain templates, and describe your policies in a simple, easily reusable way. Policy Sentry essentially describes the permissions you want in plain english, and converts it to AWS’s format. This will drastically decrease the amount of policies you see with something like
Building secure IAM policies is really, really hard. Having security teams review every single policy that is generated by the development team is not feasible – security teams are not big enough, and also this will drastically slow down application delivery.
At the same time, we cannot accept insecure IAM policies as a “fact of life”. We have the ability to capture common mistakes within the CI pipeline and stop them from ever making it to the production cloud environment. It’s on us – those practicing DevSecOps – to make that a reality.