Thursday, November 26, 2020

libusi++ shared_ptr fun

I removed my own shared_ptr<T> implementation, called ref_count<T> from libusipp. Sorry for breaking the API, but when I started the project, there was no shared_ptr<T> but now there is, and the standards version is of corse to prefer. It only comes to play when you register your own Layer2 RX or TX classes for example if you want to 'send' IP packets to a string or anything like that.

Excuse the brief README (as I just noticed), but the project is > 20y old and mainly serves internal purposes, such as qdns.

I also uploaded a new github signing key, as the old one expired.

Thursday, November 12, 2020

DoH 0-RTT trickery

I updated my DoH solution for Linux, BSD and OSX to contain more features:

* to allow certain domains to be excempted from DoH lookups and

  to be forwarded to internal DNS servers instead; in order to

  support enterprise/VPN setups where certain internal

  domains will not resolve via public DoH servers

* add 0-RTT support; unfortunately I did not find any 

  public DoH service that actually supports 0-RTT, despite some

  companies annoucing it

If you want to use 0-RTT and experiment with it, you need to build it with OpenSSL 1.1.1 or later and you need to find a DoH server supporting it. Interestingly, Cloudflare DoH servers seem to keep  TLS connections opened longer than in past. As 0-RTT only comes to play after the 1st connection by reusing TLS session tickets exchanged by the previous connection, 0-RTT will never come to play when everything works smoothly. Maybe they decided to disable 0-RTT in favor of longer lasting connections; I could not trigger 0-RTT via Cloudlfare DoH at least. If you have more infos on it, just let me know.

I also added DoH servers from switzerland to the default config, in order to distribute lookups and to avoid placing too much lookup data to the big companies.

Wednesday, September 30, 2020

More greppin speedup trickery

 I learned about SIMD based hyperscan regex scanning libs being
super fast, so I refactored grab a bit to make it possible
to load different regex engines at runtime for speed comparison.
I was also told about a quite popular similar project and
compared it to my greppin branch. Enjoy!
Still need to check whether and how it would be possible to
vectorize the matching on files to fully exploit SIMD. Will
keep you updated!
Update:I checked the code of hs_scan_vector() and it's just
iterating over the scatter array and calling internal scan
functions on it. I thought it could be using SIMD for it too,
but I was stupid. So, no more speedup on that front.

While digging into that topic, I noticed that apparently quite
lot of NIDS technology is still relying on regexes in 2020 (lol).

Monday, September 21, 2020

grep speedup trickery

I polished my parallel grep version. When I started it in 2012, multicore + SSD setups were not that common. Today, lot of storage is on flash or SSD, so you can benefit from parallel grepping by a factor of 3 or more (depending on amount of CPU cores). Just check out the link; it will also contain some timed runs to underline the statements. I also noticed that my previous git singing key expired, so I will need to resign the repos with my new GPG key (already uploaded) over time.


Update: I added a new branch to the repo to again double the speed by an dedicated nftw() + readdir() implementation thats parallelized and recursive at the same time! If you enjoy brainfucks, give it a try!





Thursday, January 9, 2020

pam_python trickery

I made a writeup about a pam_python issue
here (CVE-2019-16729 incomplete fix). pam_python is not
widely deployed, but some more fancy authentication
frameworks like face recognition on Linux seem to
require it.