Trying to build packetbeat for Raspberry PI (arm64)

Posted by On Filed under IoT, Security No Comments

After my previous article on building filebeat for Raspberry PI 3 B+ (arm64), I now wanted to get a binary for packetbeat, the second most interesting module of elastic beats. I tried the same approach with cross-compiling using GOARCH=arm64 but it fails, while a straight compile for amd64 works. It fails with a message that it excludes all Go files due to build constraints. Issue is that there is probably native C code involved and you cannot cross-compile this beat. I searched posts and tried all options for 2 hours, it does not work.

I tried again on the PI directly, the build is running but if you do a “go install”, it finally gets out of memory (“cannot allocate memory”). Problem is that the PI 3 has only 1 GB of memory and that does not seem to be enough. I tried all kinds of tricks, like setting GOMAXPROC=1, GOGC=70 but njet. The problem also seems to be related to the C build using gcc. You need to install “libpcap-dev” for the “pcap.h” header file using “apt-get install”, otherwise one gets a compile error earlier.  If using “go build -v -x” directly you get this “cannot allocate memory” message from gcc. When using “make” the build gets killed instead. Nevertheless it’s rare are there are reports from people that compile Kubernetes on arm64 RPI 3B like mine. Probably K8s does not contain native C parts like the libpcap in packetbeat. So I finally gave up, because …

But … there is good news ahead! Since a few days, the new Raspberry PI 4 Model B has been released! With up to 4 GB memory that will hopefully work. Also it has now true GB LAN, which is for network sniffing, not a bad idea either, when attaching it to a real network tap. So that is a clear buying plan for Juli!

Installing filebeat on Raspberry PI 3 (amd64)

Posted by On Filed under IoT, Security, Technology, Uncategorized No Comments

Currently I’m experimenting with using a Raspberry PI 3 B+ as a network security monitoring (NSM) sensor node. So I have Bro and Suricata installed on that little guy running Kali Linux for arm64. But I need a modern way to transport the logs to its log monitoring station. So not using syslog-ng or ryslog but the best log shipper for the elastic stack, and that is Beats, better the Filebeats.

Problem: Elastic does, unfortunately, despite desperate inquiries from users in the forums not provide binaries or a .deb package for Beats. After trying some other paths I came across some receipts to install Beats on arm64 by manually compiling the binary with Go. I have to say, Go is marvelous! On the PI itself, I had bad luck, because the “go build” quickly finished with out-of-memory. So that didn’t work unfortunately on that little pal.

But because Go is so cool, I just “cross-complied” it on a bigger laptop, also running Kali Linux. And that’s so easy that I have to tell the world, because the other receipts are sometimes too specific and parts are missing for a full running manual installation, which is more than just the filebeats binary.

Step one on the other Debian-base system, the laptop, you need of course also Go installed.

# mkdir -p go/src/github.com/elastic
# cd go/src/github.com/elastic
# git clone https://github.com/elastic/beats.git
# export GOPATH=$PWD/go

You could also get the sources with “go get” bu that doesn’t matter, result the same.
Now the important step, watch out:

# export GOARCH=arm64
# cd beats
# go build -v -x

Flags just so see what’s happening, as go build is very silent otherwise. Magic, in a few seconds, you have a “filebeat” binary in this directory!
Try:

# file filebeat
filebeat: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=svVi8LJGhqXEjRJveTrA/7cOYouMPn1VzyeJqwq3W/TXZ3DZ8Wa_QYdKnsR8cm/8bg35yoawYw18mAJ30oX, not stripped

Remember we are on amd64 not arm64 on the laptop!
Now just copy the file over to the PI using ssh and test it there:

# ./filebeat –help

Works! But when you try

# ./filebeat modules list

It does not show any, because we are missing something, all the module configuration and kibana dashboards that are normally also contained in the .deb package.  So on the laptop just install filebeats, as for amd64, there is of course package:

# cd ~/Downloads
# curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.1.1-amd64.deb
# dpkg -i filebeat-7.1.1-amd64.deb
# filebeat modules list

Here you get the modules of course.
Now just let’s see what’s in the debian package:

# dpkg –listfiles filebeat|more

As you can see, besides the binary (for amd64) no other binaries are really in the .deb, just lots of YAML and JSON files. Now that’s of course good news.
So what I did for getting a fully functional installation is just copy the files over form the laptop to the PI using SSH in /etc/init.d/filebeat, /etc/filebeat/*, /usr/share/filebeat, /lib/systemd/system and /usr/bin/filebeat (a script). Then place the compiled arm64 binary in “/usr/share/filebeat/bin/filebeat” and we’re got to go on the PI:

# filebeat modules list

And here we get the list.
Now this is not a package that will be manged by apt-get of course. Maybe, I didn’t try one could for to install the official amd64 .deb package and only exchange compiled binary.

Hope this helps, Peter