Introduction
If you have spent any time reading some of my older articles, you know I am a fan of jq. In my article on How to Exploit APIs with cURL I showed how to parse API responses with it. I went even further when I showed how to extract artifacts from OpenAPI docs using jq to help attack APIs.
The thing is, I get that jq can be overwhelming… to the point of being far too difficult to use if you are just starting out. And there are easier ways to parse JSON data, especially when you don’t really know the structure of a data object for a new API endpoint your stumble across.
In this article, I will show you how to grep API payloads with gron.
What is Grepping?
Grep stands for “global regular expression print” and is a command line utility used to search through text files and data. It’s very useful for narrowing down large data sets when you know the type of data you are looking for but doesn’t necessarily know where to look for it in the file.
Think of it as an advanced search engine for plaintext.
When you hear someone saying they are “grepping” through data, it might not actually be grep that is used. You could just as easily use sed or awk to do it. The use of “grepping” is just shorthand for saying you are searching through a data set.
When working with an API though, it’s not really plain text in your data set. It is usually JSON objects. This is where the power of Gron comes in.
What is Gron?
Gron (which stands for grepping JSON) is a command line utility written in Go to make grepping through JSON easier. It was written by fellow security researcher Tom Hudson (aka @tomnomnom) and can be found in his gron GitHub repo here.
In this article, I will show you how it works and how it can be used with an API response payload to parse out the information you need quickly and easily.
It works by taking in an input file or object (in this case, a JSON object), flattening the structure, and then exposing all the values as simple key/value pairs, which can then be searched using regular expressions. This makes grepping through complex data objects from an API endpoint much simpler.
Let’s get right into it.
Using Gron to Parse API Payloads
The best way to talk about Gron is just by demonstrating it. It really is that easy of a tool to use.
Fetching API Data
Let’s start by working with an API endpoint we all can access. Since everyone likes to talk about cryptocurrency these days, let’s imagine we want to get the current Bitcoin Price Index. Coindesk exposes an API for that at api.coindesk.com.
Let’s pull it using cURL:
curl -s https://api.coindesk.com/v1/bpi/currentprice.json
The data returned is messy. Luckily this payload is pretty small, so it’s not too hard to understand. But you can see how ugly this gets fast when you start pulling back larger and more complex data objects.
Enter in Gron.
Grepping with Gron
So first, let’s make it easier on us to read the payload using Gron:
curl -s https://api.coindesk.com/v1/bpi/currentprice.json | gron
As you can see, it becomes much easier to read when Gron flattens the JSON object and sets it into key/value pairs.
Now let’s add grep into the mix. Let’s imagine we wanted to get the current value in Euro for a Bitcoin. That isn’t too hard now:
curl -s https://api.coindesk.com/v1/bpi/currentprice.json | gron | grep EUR
Nice! We can see the data for Euros. They actually have the rate as a float that we can use in some sort of math function. Let’s extract that and dump it into a variable called PRICE that we can use later.
PRICE=$(curl -s https://api.coindesk.com/v1/bpi/currentprice.json | gron | grep EUR.rate_float | cut -d ' ' -f 3 | tr -d ';')
echo $PRICE
Ahhhh ya. Cracking open a JSON payload couldn’t be easier. With a bit of code, you can pretty much extract any data you need in a pretty straightforward format.
Transforming data back to JSON
Now here is a nice touch Tom added to Gron, which makes tampering with data kinda fun. You can reverse gron formatted data and shove it back at the API if it supports POST or PUT operations. You just use the –ungron argument.
So imagine we wanted to construct a partial JSON object to send back to the API. It’s as easy as:
curl -s https://api.coindesk.com/v1/bpi/currentprice.json | gron | grep EUR | gron --ungron
Notice you have a partial JSON structure of the child nodes you grepped out. If you can POST or PUT that back to the API, you have a full-circle operation to tamper with captured payloads. You could easily script something in Bash or Python to do input injection in specific properties and attack an API by tainting data in weird places.
I love it when a plan comes together.
Advanced Gron Usage
So I watched a fascinating talk at NorthSec 2023 on Tips & Tricks for BurpSuite Pro by Nicholas Gregoire. He demonstrated an interesting way to use the Piper extension in BurpSuite to bring gron right into your BurpSuite output.
Check this out:
Consider the uses for this. Or any other command line tool you would like to pull into BurpSuite. Heck, you could even pull in jq in the same manner. Pretty powerful stuff!!
Conclusion
When it comes to hacking APIs and parsing through large and complex data objects, Gron is an excellent tool for your toolkit. Its ability to flatten JSON and expose key/value pairs for easy searching using regular expressions makes it incredibly powerful and efficient.
Furthermore, Gron’s ability to reverse its formatted data back to JSON and insert it back into the API payload for tampering purposes is a game-changer in terms of input injection and API attacks.
Whether you’re a seasoned security researcher or just starting out in the world of hacking, Gron is a tool that can simplify your workflow and make your life easier, especially if you find jq too complex. So if you’re looking to improve your API hacking skills, be sure to give Gron a try. You won’t be disappointed.
Now, go forth and grep with gron!
Like what you’re reading? Then make sure you subscribe to the API Hacker Inner Circle newsletter to keep up with my work and what is happening in the community.
