More PSC trickery!!

I updated psc to include SOCKS4 and SOCKS5 support, so you can now do crazy things like web browsing remote networks from within a modem dialup shell or portshell (even multihop), effectively upgrading a simple portshell to a SSH like e2e pty shell with the ability to forward TCP and UDP ports.

This finally merges code into PSC that I started back in early 2000's, when I was in need to have TCP connections via modem dialups that actually did not have ppp to obtain an IP address to browse from.

Port Shell trickery

 

Added new feature to my old long running project psc. You may now forward TCP or UDP ports in a similar way as with ssh -L. The cool thing: You don't even need an IP address or network connection to the remote hop. An UART or modem connection will suffice. As long as you have a tty session, you can now slip TCPv4, UDPv4, TCPv6, UDPv6 through it and appear with your connections as if they were made on the remote end.

A demo video is on asciinema.

libusi++ shared_ptr fun

I removed my own shared_ptr<T> implementation, called ref_count<T> from libusipp. Sorry for breaking the API, but when I started the project, there was no shared_ptr<T> but now there is, and the standards version is of corse to prefer. It only comes to play when you register your own Layer2 RX or TX classes for example if you want to 'send' IP packets to a string or anything like that.

Excuse the brief README (as I just noticed), but the project is > 20y old and mainly serves internal purposes, such as qdns.

I also uploaded a new github signing key, as the old one expired.

DoH 0-RTT trickery

I updated my DoH solution for Linux, BSD and OSX to contain more features:

* to allow certain domains to be excempted from DoH lookups and

  to be forwarded to internal DNS servers instead; in order to

  support enterprise/VPN setups where certain internal

  domains will not resolve via public DoH servers

* add 0-RTT support; unfortunately I did not find any 

  public DoH service that actually supports 0-RTT, despite some

  companies annoucing it

If you want to use 0-RTT and experiment with it, you need to build it with OpenSSL 1.1.1 or later and you need to find a DoH server supporting it. Interestingly, Cloudflare DoH servers seem to keep  TLS connections opened longer than in past. As 0-RTT only comes to play after the 1st connection by reusing TLS session tickets exchanged by the previous connection, 0-RTT will never come to play when everything works smoothly. Maybe they decided to disable 0-RTT in favor of longer lasting connections; I could not trigger 0-RTT via Cloudlfare DoH at least. If you have more infos on it, just let me know.

I also added DoH servers from switzerland to the default config, in order to distribute lookups and to avoid placing too much lookup data to the big companies.

More greppin speedup trickery

 I learned about SIMD based hyperscan regex scanning libs being

super fast, so I refactored grab a bit to make it possible

to load different regex engines at runtime for speed comparison.

I was also told about a quite popular similar project and

compared it to my greppin branch. Enjoy!

Still need to check whether and how it would be possible to

vectorize the matching on files to fully exploit SIMD. Will

keep you updated!

Update:I checked the code of hs_scan_vector() and it's just

iterating over the scatter array and calling internal scan

functions on it. I thought it could be using SIMD for it too,

but I was stupid. So, no more speedup on that front.

While digging into that topic, I noticed that apparently quite

lot of NIDS technology is still relying on regexes in 2020 (lol).

grep speedup trickery

I polished my parallel grep version. When I started it in 2012, multicore + SSD setups were not that common. Today, lot of storage is on flash or SSD, so you can benefit from parallel grepping by a factor of 3 or more (depending on amount of CPU cores). Just check out the link; it will also contain some timed runs to underline the statements. I also noticed that my previous git singing key expired, so I will need to resign the repos with my new GPG key (already uploaded) over time.

Update: I added a new branch to the repo to again double the speed by an dedicated nftw() + readdir() implementation thats parallelized and recursive at the same time! If you enjoy brainfucks, give it a try!

pam_python trickery

I made a writeup about a pam_python issue
here (CVE-2019-16729 incomplete fix). pam_python is not
widely deployed, but some more fancy authentication
frameworks like face recognition on Linux seem to
require it.

D'oH! no-IPv6-workaround trickery

I added a workaround to my harddns DoH project. Some browsers accept AAAA records and prefer them over A records, even when there is no IPv6 connectivity. Inside the NSS module, it is hard to distinguish these cases and its certainly not our task to check for IPv6 connectivity during DNS resolves. Therefore, I added a nss_aaaa config flag, which needs to be enabled if you want the NSS module to lookup AAAA records for certain gethostbyname() calls. Certain - because within various incarnations like gethostbyname2(), getaddrinfo() etc., some of these functions lead to NSS ‘backend’-functions which can make the case by checking an address-family parameter. Some functions can’t and will use the nss_aaaa helper flag from the harddns config. It is disabled by default, so if you use harddns via the NSS config, you need to enable it, if you have IPv6 connectivity and want to profit from it. The DoH proxy server works as before, since its just translating between DNS and DoH back and forth.

IPv6 massbind trickery

Some more open-source community love for handling of lot of ip6 addresses on a single interface, to pick and dispose per connection for example to evade address based statistics:
here.

I also cleaned up this' blog link-list to free it
from dangling links and adding some new which I found
interesting to read.

Hey folks, if you are wondering what I may be doing, I am still
splitting my time in coding and reading of other ppls code.
Here's some results.

I analyzed quite some DoH "solutions" for mobile and desktop (browsers),
and I can tell you that most of them are really bloated. What do you think,
how many TCP/IP stacks you can stack on an Android phone? If you think
you may be just running one or two: good chances that you may be wrong.

As it turned out even malware authors jump onto the DoH train, but
nevertheless the experts are discussing whether its really DoH or not.

OTOH, I used the time to add some new stuff to my own DoH
repo.

The fancy new features are:

 * Support for BSD and OSX
 * caching daemon
 * support for rfc8484
 * supports all major DoH providers out of the box

Sure, nothing that the big tech companies offer you is for free, not
even DoH! So apparently even your DNS requests are of interest. So you
have to decide yourself whether you accept DNS redirects by your gov
to inject some malware, or allow tech companies to play big data
with your lookups.

After all, porting harddns or any other non-trivial code to BSD,
you gotta love -pedantic. Still, BSD has its special needs and
its always good to know.