OWASP SecurityRAT

Posted by On Filed under Security, Technology No Comments

SecurityRAT is a OWASP open-source project (github), the RAT stands for requirements automation tool. Currently the version 1 (1.7.8 as of the time of writing) is the productive version, but a version 2 is in the making with new architecture. SecurityRAT is based on JHipster Java rapid application development framework. Version 1 is a classic application, version 2 will be JHipster micro-service based.

SecurityRAT is used to create a set of security requirements for a supporting asset, the main part of a security concept. In the end it is nothing more than a replacement for MS Excel a way to get a filtered list of requirements with status and details provided by the development team on the implementation of the requirement in the asset. That status and the details are the important part, because having a list of requirements if fine but without knowing, whether they have been implemented or not and how, they don’t help a lot. Spending time on the details is important, they should contain additional information on the implementation, such as a link to the Jira or TFS issue, a link to a wiki page with implementation details or prove of implementation.

The cool thing is that one can define its own fields and values as long as it fits into the general idea of SecurityRAT. The central concept is a list of requirement skeletons classified by categories and tags that have columns and belong to a project type. The requirements in the database are skeletons or templates, from which instance of requirements for a given supporting assets will be created at run time. Those requirement instances are never actually stored in the database but exist only in memory on the client (browser) side. As soon as one has understood this, it’s a done deal in understanding how the tool works. You pour a pre-classified list of requirement templates into a database and instantiate them for a supporting asset at runtime with the additional benefit of filtering the list down to the relevant ones using category questions, filtering and tag selection at runtime.

We use SecurityRAT as an expert tool. This means, that not all developers work with the tool all day but only selected security lead experts, do. SecurityRAT spits out a Excel document with the requirement instances together with the up-to-date status and comment (optional columns). This is what other people work with. You put the Excel into the wiki for documentation or generate sub-tasks in Jira etc. SecurityRAT can also directly create stories and sync them bi-directionally, which would be really cool. Unfortunately this doesn’t work at my place, don’t ask why, it’s a sad big enterprise problem.

But working with Excel, or better CSV files, has some advantages, too. You can convert it to markup or generate task language in Jira from it easily with a little script. I use Groovy for it, but that’s a matter of personal taste.

SecurityRAT comes out of the box with a SQL dump for the OWASP ASVS (Application Security Verification Standard) requirements catalog. We have in the mean time at work also version 4.1 and many other catalogs that we pour into SecurityRAT instances. Version 1 somehow requires one RAT instance per catalog, although you can of course put multiple catalogs that have the same structure into one big instance, e.g. CIS Benchmarks. That ends up in a big list of instances for e.g.

  • OWASP ASVS
  • IEC 62443
  • DIN SPEC 27072
  • Corporate security requirement lists
  • CIS Benchmarks (you need to be member to get the XLS files)
  • Own catalogs e.g. for RabbitMQ

Apart from really filling in security requirements, SecurityRAT can be mis-used very well for other tasks. Things I use it for are among others:

  • Vulnerability assessment according to the OWASP Testing Guide (OTG). Excellent, you set the status to passed/failed and fill in the findings and get a nice Excel
  • Security maturity assessment according e.g. BSIMM or OWASP SAMM, answer the questions pre-filtered by level that should be achieved and get a nice Excel
  • Threat modelling using STRIDE – that stretches a bit the idea but works, when you have a list of threat skeletons instead of requirements.

Using SecurityRAT for status tracking with the OTG or ASVS are a good example where it makes sense to fill multiple, testing or requirement guides, into one instance, e.g. web services, mobile and IoT. The make a category or project types for these (one support asset can be either a web service or a mobile app or a IoT device). This way the user selects the type of asset in the initial “question” aka collection instance list implicitly.

In the end you could do a lot of this with Excel but you have IMHO the following advantages by SecurityRAT:

  • It’s not a document that rots somewhere on a share but a server with a nice web-based interface
  • The definable collection categories allow you to pre-filter the requirements at the beginning using customizable questions. Yes you could filter in Excel but you don’t have this very usable two-step process that helps in reality. Using Tags you can also filter when the requirements list has been generated, as well.
  • You can save the working results in a YAML file, load that again and continue, e.g. by adding also custom requirements. So only one place.

Being a server and database solution, filling an instance with data could either be done by UI, but you will quickly skip this idea for larger catalogs, even when the batch operations are really handy. Just use (again Groovy or other) scripts to convert a CSV source into SQL statements or directly insert it into the DB, which is then a bit more work. Unfortunately the entities in the SecurityRAT do not have surrogate keys so your script needs to manage the uniqueness of the database IDs by itself, which is sometimes, well a mess.

The down-side of the tool is a bit the missing calculation and missing colors for status fields that you have in a spreadsheet. E.g. for risk assessments it would be cool to calculate a risk factor from likelihood and impact automatically. But that is not possible.

BTW, SecurityRAT runs with docker out of the box, using MySQL or for license sake MariaDB is no problem. Problems will manifest themselves with endlessly long Spring Java exceptions, that will require a bit of digging into. We run everything in docker-compose, backup with mysqldump using docker exec and a nice landing page for the different instances.

Overall SecurityRAT, thumbs up!

HTTP Health Check for Docker

Posted by On Filed under Security, Technology No Comments

I just published a little tool called htcheck (docker-health-go project) on github: https://github.com/pklotz/docker-health-go . This being my first Go program, be kind to me, should there be better ways to implement it. 

It is intended as a very simple HTTP health check client for use in Docker health checks. Motivation, why this program?

When using docker directly or via docker-compose, you can and should define a health check, so that docker knows that the process it is running is doing well. There are a couple of libraries to provide a HTTP health endpoints for Go, such as [https://github.com/docker/go-healthcheck] and also Java offers with Spring Boot Actuator corresponding framework.

But on the client side, you still need to use curl or the outdated wget to perform the check. If you ever checked, which dependencies curl and thus libcurl4 brings with it, you might wonder if this is worth the ballast just to do a simple HTTP get with an exit code. Libcurl brings openldap libraries into the image and what not. So this little decent project provides a special-purpose HTTP client to use for health checks in docker or elsewhere instead of throwing a general-purpose HTTP client at the job.

It supports making a HTTP GET request to a URL and reading a JSON document back and checking for a value in it using a jq-like path expression.

Simple sample usage in Dockerfile:

COPY ./htcheck /usr/bin/

HEALTHCHECK --interval=5m --timeout=3s CMD htcheck -u http://localhost/ || exit 1

Sample usage for Spring Boot actuator health endpoint, which normally serve a JSON document in the form:

{
    "status" : "UP"
}
So using the JSON path feature, we can compare against a expected value:
COPY ./htcheck /usr/bin/

HEALTHCHECK --interval=5m --timeout=3s CMD htcheck -u http://localhost/health -p .status -v UP || exit 1

Licenses are checked and documented in the README. Thanks to the dependency projects [https://github.com/savaki/jq] and [https://github.com/spf13/pflag] that were made use of. Probably some features are still missing, but as a first shot, it should serve. S’il vous plaît!

Installing filebeat on Raspberry PI 3 (amd64)

Posted by On Filed under IoT, Security, Technology, Uncategorized No Comments

Currently I’m experimenting with using a Raspberry PI 3 B+ as a network security monitoring (NSM) sensor node. So I have Bro and Suricata installed on that little guy running Kali Linux for arm64. But I need a modern way to transport the logs to its log monitoring station. So not using syslog-ng or ryslog but the best log shipper for the elastic stack, and that is Beats, better the Filebeats.

Problem: Elastic does, unfortunately, despite desperate inquiries from users in the forums not provide binaries or a .deb package for Beats. After trying some other paths I came across some receipts to install Beats on arm64 by manually compiling the binary with Go. I have to say, Go is marvelous! On the PI itself, I had bad luck, because the “go build” quickly finished with out-of-memory. So that didn’t work unfortunately on that little pal.

But because Go is so cool, I just “cross-complied” it on a bigger laptop, also running Kali Linux. And that’s so easy that I have to tell the world, because the other receipts are sometimes too specific and parts are missing for a full running manual installation, which is more than just the filebeats binary.

Step one on the other Debian-base system, the laptop, you need of course also Go installed.

# mkdir -p go/src/github.com/elastic
# cd go/src/github.com/elastic
# git clone https://github.com/elastic/beats.git
# export GOPATH=$PWD/go

You could also get the sources with “go get” bu that doesn’t matter, result the same.
Now the important step, watch out:

# export GOARCH=arm64
# cd beats
# go build -v -x

Flags just so see what’s happening, as go build is very silent otherwise. Magic, in a few seconds, you have a “filebeat” binary in this directory!
Try:

# file filebeat
filebeat: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=svVi8LJGhqXEjRJveTrA/7cOYouMPn1VzyeJqwq3W/TXZ3DZ8Wa_QYdKnsR8cm/8bg35yoawYw18mAJ30oX, not stripped

Remember we are on amd64 not arm64 on the laptop!
Now just copy the file over to the PI using ssh and test it there:

# ./filebeat –help

Works! But when you try

# ./filebeat modules list

It does not show any, because we are missing something, all the module configuration and kibana dashboards that are normally also contained in the .deb package.  So on the laptop just install filebeats, as for amd64, there is of course package:

# cd ~/Downloads
# curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.1.1-amd64.deb
# dpkg -i filebeat-7.1.1-amd64.deb
# filebeat modules list

Here you get the modules of course.
Now just let’s see what’s in the debian package:

# dpkg –listfiles filebeat|more

As you can see, besides the binary (for amd64) no other binaries are really in the .deb, just lots of YAML and JSON files. Now that’s of course good news.
So what I did for getting a fully functional installation is just copy the files over form the laptop to the PI using SSH in /etc/init.d/filebeat, /etc/filebeat/*, /usr/share/filebeat, /lib/systemd/system and /usr/bin/filebeat (a script). Then place the compiled arm64 binary in “/usr/share/filebeat/bin/filebeat” and we’re got to go on the PI:

# filebeat modules list

And here we get the list.
Now this is not a package that will be manged by apt-get of course. Maybe, I didn’t try one could for to install the official amd64 .deb package and only exchange compiled binary.

Hope this helps, Peter

Projango Agile

Posted by On Filed under Technology No Comments

At blue elephant, we are using, SCRUM as our development methodology. Now Edgar, my project lead would scream and cry that we would not do it right, but we are doing agile development SCRUM-style, for me that is just fine. Doesn’t have to be the pure law 🙂

As the SCRUM tool of choice we use Projango the SCRUM project management of Xenatec. By pure coincidence Edgar is partner of Xenatec and co-author of Projango. Blue elephant has been, so to say, the major beta tester for Projango in the last year(s) and in the mean time this tool is our core tool for the development team.

The visually very nice Projango SCRUM-board is in daily use by the developers to work on their tasks so that everybody knows who is doing what as well as for time bookings. Using the board we do task break downs of the user stories and effort estimations in the team. Instead of someone writing the tasks in an excel to centrally enter them in a system, we immediately create the tasks, prioritize and estimate them online during our planning sessions.
Urgent customer requirements are entered in the backlog and linked via Weblinks with our bug tracking tool Jira.

The big advantage of Projango versus other SCRUM solutions, e.g. from Atlassian, is the very direct visual manipulation of stories and tasks. Just drag and drop them to prioritize or change the state. In-place editing and the engineer-images are very cool, so that one visually sees who’s working on or verifying a task. Also the impressive burndown chart and the generated MS Excel documentation help to manage and present R&D work in the organization.

We can only recommend Projango, so have a look!

IntelliJ IDEA

Posted by On Filed under Technology No Comments

Can’t code without Since years now I work with IntelliJ IDEA as my Java IDE, which I introduced already back at HP as the Java development tool of choice. While there had been some small problems with performance in earlier versions, it got now better and faster again than ever before.
Meanwhile the Eclipse users around me are getting more and more and they brag about why it would be so superior and much better. I just let them talk and lough at them, because in the end I’m still so much more productive with IDEA. And if I’m the last IDEA user, I will never surrender to Eclipse, which is full of over-engineered features, usability nightmares and unnecessary complexities.
I just don’t understand it why the do not prefer IDEA, where do some people look at? Are they running blindly through the world or just behind the masses? Development is like handcraft, you need the right tools for the work, then it is already half done. IDEA is the powertool for Java and Groovy developers, unparalleled in this ecosystem. At least for me, even if I have given up on these Eclipse-guys meanwhile.

Gradle

Posted by On Filed under Technology No Comments

For all my professional career, I have been involved in software build systems, starting with Make, Imake, Ant, Maven etc. Currently we use Ant at blue elephant but the build is huge, hard to maintain and totally unmodular. No point, that one could also do it better in Ant or let yourself subjugate on the rigid conventions of Maven. But what I was looking for is to minimize efforts of migration, be flexible to do things a bit adapted but still have the nice convention based build and especially the dependency management of maven.
It turns out that there is indeed a optimal solution for it and it comes with the name Gradle. We are anyhow using a lot of Groovy (see earlier posts), which is for me the optimal scripting language. Gradle is well documented, in contrast to the ever mystic maven, it is non-XML using a Groovy DSL as notation and it can use Ant tasks and scripts, which eases migration a lot. And anyhow, there some things in Ant that are unparelleled elsewhere, like the fileset-based operations and there are tons of Ant tasks out there, like xmltask, scp etc. that it would be dump to re-implement just for the fun of it.

Also there seem to be a lot of enthusiastics for Gradle, like this blog. And there is one fan more, so we will soon see some Gradle in MIDAS probably …

Java on MacOSX

Posted by On Filed under Technology No Comments

I’m an Apple enthusiast, really in nearly all respects. I have a iMac, a MacBook, a iPhone and hopefully some time an iPad, I have no worries about the app store with either iOS nor MacOSX, not about their unti-Flash war, I pay their prices, watch their announcements, all, really I’m convinced.
But this message about Apple seeking to get rid of their Java Development Kit is too much. Java is not Flash. Java is the center of my world as a developer and architect. Java is the most important ecosystem and implementation system in the world. To try to drop that on the MacOSX platform is insane, mad, unbeliefable and stems from an unparalleled arrogance and superstition. Someone at Apple seems to run amok or has a malfunction in the brain, I can only hope that this isn’t Steve’s idea.
Hello dudes, all server software is Java, tons and tons of desktop tools, heard of Eclipse, Netbeans, IntelliJ IDEA? Does Tomcat, Jetty, Servicemix sound familiar? Any idea of what you are talking about? About the end of MacOSX as a developer platform, as a server platform, shall I continue?
You better think about that idea once more you crazy idiots at Apple!

Scripting languages on the JVM

Posted by On Filed under Technology No Comments

Since I while I’m a fiery fan of Groovy, but as usually you hear form some persons, oh Scala is cool, or Ruby or Python or, or, or. So I wanted to know whether I have developed a certain bias towards Groovy or there is some good reason behind this intuitive selection. So I have in the last time made some effort to look into other scripting languages especially on the JVM. I had some exposure to scripting languages before, first REXX in my old days on VM/CMS or Phython/Jython as one of the first object-oriented scripting languages, Tcl/Tk or of course Unix shell including awk/sed etc.

My personal conclusion to the hype about Scala e.g. is for the time being, that it is a very very large language system which is probably very hard to learn, harder than Groovy and honestly I dislike somehow some of the syntactical elements of Scala, which are somehow just different from the Java/Groovy conventions without some obviously good reason. I’m a fan of functional programming, did some real Lisp on Emacs in my old days at the University of Stuttgart. But the mix of functional and procedural programming paradigms in Scala are somewhat not as cool and urging as in Groovy IMHO. Scala seams to be something of all but nothing really right. For example, take Clojure, a purely Lisp-style functional scripting language. Pure and nice, less braces than Lisp but still somehow simple to learn, a beauty in comparision to Scala. Or the closures in Groovy, really really nice to use, so superior in my eyes.

And Groovy has the absolute top advantage that its Syntax is so close to Java’s and you can quasi 100% integrate Groovy into and onto Java, mix and match. That is exactly what one needs if looking for a application extension and scripting language, no other candidate can do this in the way Groovy does. Inherit from a Java class or vice versa, inject methods from Groovy into existing Java classes to make then feel Groovy, really really cool!

That is why we do in MIDAS provide a API in Groovy on top of our Java GUI. I am working though additionally on a Clojure API wrapper, just for fun though.

XMLTask Ant Task

Posted by On Filed under Technology No Comments

In MIDAS we use ant as one of our core techniques and also apply a couple of third-party tasks in addition, such as ant-contrib. I have written and contributed for example the ant tasks for the eXist XML database and have a extensive experience in writing custom ant tasks. That’s why I especially appreciate the XMLTask a task for XML modification and creation from OOPS Consultancy Ltd, a UK-based consulting company.

While a lot of jobs can be handled by specific XSL stylesheets, it always needs an extra XSL file in addition to the ant task ant doing small things like insert some tag or attribute, remove a portion etc. a direct XPath-based XML manipulation is much faster to write with the xmltask than writing templates in an XSLT. And the cool thing about the xmltask is that it can put the result of a XPath query into a buffer that can then e.g. be re-inserted into another document without writing it to a file or such, just a ant-internal object just like a fileset. So far I haven’t found anything that couldn’t be done with this task regarding XML processing, it even supports DTD catalogs and schemas, maintaining the DTD declaration etc. pp.

Most definitively a must look at!