File retrieval with Osquery using carves on Zercurity

SELECT
*
FROM
carves
WHERE
path LIKE '/Users/tim/Downloads/%'
AND carve = 1

Example

Zercurity is free to deploy locally and free to test online. The GIF below shows you a quick query to lift any and all files found within any Users download fonder on Mac OSX.

Downloading files from a users Download folder using Osquery Carves on Zercurity.

Extracting the archives

To open your carved archive you’ll need to use either untar or tar to extract the archive.

untar bundle.tar  # extract TAR archive
tar -I zstd -xvf bundle.zst # extract ZST archive
sudo apt install zstd

Osquery carver remote server settings

In order to get carving working you will require a remote Osquery server. You can download an example server here. Or you can spin up Zercurity using docker-compose here (which is configured with carving enabled by default).

--disable_carver=false
--carver_compression=true
--carver_block_size=5120000
--carver_start_endpoint=/start
--carver_continue_endpoint=/upload
--carver_disable_function=false
--tls_hostname=api.zercurity.local
--enroll_always=true
--enroll_tls_endpoint=/v1/enroll
--enroll_secret_path=/tmp/enroll_secret.dat
--config_plugin=tls
--config_tls_endpoint=/v1/config
--distributed_plugin=tls
--distributed_tls_read_endpoint=/v1/distributed/read
--distributed_tls_write_endpoint=/v1/distributed/write
--distributed_interval=600

Osquery carver server

There are two important resources that are needed in order for the client to upload data back to server.

The /start resource

The start resource is initially hit by the client to retrieve a session token and let the server know how many blocks will be uploaded the server. As the archive is chunked up into blocks as specified by the carver_block_size. The requesting payload is as follows:

{
"block_count": 17,
"block_size": 5120000,
"carve_size": 878610143,
"carve_id":" f47cfd64-4750-449b-a78d-2b631f317ae4",
"request_id": "5028338a-2461-41c6-b6e3-1414ec78b208",
"node_key": "hFTdv8F...y39YkKaEC"
}
{
"session_id": "qhKvSKME...gxEEw37Z"
}

The /upload resource

Once the /start request has completed and a session_id returned. The client will start to upload the necessary blocks. The payload for each request will be as follows:

{
"block_id": 4,
"session_id": "qhKvSKME...gxEEw37Z"
"request_id": "5028338a-2461-41c6-b6e3-1414ec78b208"
"data": "aDZYWmV3dUJ0NW..FaczlmaGYZlgyVUQ="
}

Finishing up

The Osquery client will not post a final request to let the server know it has finished uploading all the blocks. Instead the server needs to keep track of all the blocks uploaded. Once all the blocks have been received. Matching the final block_count . All the uploaded blocks can be stitched back together to form the final archive.

file_ext = 'tar'
file_size = 0
fd, path = tempfile.mkstemp()

try:
f = open(fd, 'wb')
for block_id in range(0, carve.block_count):
raw_data = get_block(carve.uuid, block_id)
file_size += len(raw_data)
if block_id == 0 and raw_data[0:4] == b'\x28\xB5\x2F\xFD':
file_ext = 'zst'
f.write(raw_data)
f.close()
except Exception as e:
raise e
finally:
return path, file_ext, file_size

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Zercurity

Zercurity

Real-time security and compliance delivered.