Capturing Osquery query results with AWS Firehose (Kinesis) and AWS Athena

Why?

How?

--aws_access_key_id "ACCESS_KEY"
--aws_secret_access_key "SECRET_KEY"
--aws_region "us-east-1"
--aws_kinesis_stream "osquery"
--aws_firehose_stream "osquery"

Configuring AWS Kinesis

AWS S3

Creating our S3 bucket for our Osquery Firehose logs.

Kinesis

Creating our Osquery Firehose for “Direct PUT or other sources”
Choosing our S3 bucket for our AWS Kinesis Firehose
Create our IAM role for our Kinesis delivery stream.

Creating our Kinesis IAM policy

Creating our programmatic user

Creating our AWS Kinesis user
Attaching our Osquery Firehose policy
Downloading our API keys for Osquery to access the AWS Firehose API

Configuring Osquery to send data to AWS Kinesis Firehose

/usr/local/zercurity/zercurity.pem  # Mac OSX
/opt/zercurity/zercurity.pem # Linux
C:\Program Data\zercurity\zercurity.pem # Windows
Exception making HTTP request to URL (https://firehose.eu-central-1.amazonaws.com): certificate verify failed
openssl s_client -showcerts -verify 5 -connect firehose.eu-central-1.amazonaws.com:443
cat /etc/ssl/certs/ca-certificates.crt

Updating your Osquery configuration

/Library/LaunchDaemons/com.zercurity.osqueryd.plist  # Mac OSX
/etc/osquery/osquery.flags # Linux
C:\Program Data\zercurity\osquery\osquery.flags # Windows
--aws_access_key_id "ACCESS_KEY"
--aws_secret_access_key "SECRET_KEY"
--aws_region "us-east-1"
--aws_firehose_stream "osquery"
# For Mac OSX run the following from the command line to restart the Osquery servicesudo launchctl unload \
/Library/LaunchDaemons/com.zercurity.osqueryd.plist
sudo launchctl load \
/Library/LaunchDaemons/com.zercurity.osqueryd.plist
# For Linux. Depending on your distribution you can run one of the following:sudo systemctl restart osqueryd
sudo /etc/init.d/osqueryd restart
# Windows Start->Run. Launch services.msc and then restart the Osquery serviceservices.msc
AWS Osquery Firehose results in AWS S3
{
"name":"f64af85f-a05e-4601-98cd-6c9a8f35feec",
"hostIdentifier":"MacBook-Pro.local",
"calendarTime":"Sun Feb 28 09:40:01 2021 UTC",
"unixTime":"1614505201",
...

"columns: {
"action":"CONNECT",
"family":"2",
"local_address":"192.168.15.249",
"local_port":"62531",
"path":"\/usr\/bin\/ssh",
"pid":"78775",
"protocol":"6",
"remote_address":"192.168.31.248",
"remote_port":"22",
"timestamp":"1614456741"
},
"action":"added",
"log_type":"result"
}

AWS S3 Athena

Creating our AWS Glue crawler
Configuring our AWS Glue crawler
Giving the AWS Glue crawler access to our S3 bucket containing our Osquery result data
Running our new on-demand AWS Glue crawler
Querying Osquery sockets data with AWS Athena

Its all over!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store