Thursday, May 4, 2017

OpenSSL constant hell

Last week, someone opened a bug for opmsg, saying it
wont compile with recent versions of gcc.
I am happy that I've been noticed about it, since it shows
people are using opmsg. The "bug" however is not within
opmsg, its about the way OpenSSL changes their function
definitions (breaking the API!) in between minor versions
of OpenSSL.

What exactly happened?

In above screendump you see two functions constp() and
constpp(). They will serve as an easy, down-stripped demo.

While with constp() the const char * declaration
by the programmer is more like a promise to the caller
that the data at which ptr points to, will not be written to,
this is different when the type changes to a double
pointer. You can pass char * variables to constp() without
any problem, because constp() just promises you to not modify
the pointee data. It would be allowed to do so for char *
variables, but it confines itself to this promise, which
is good practise to show the user of the API "Hey, we
wont modify your data as you pass it to us." 
There are no problems if you would change a foo(char *) 
declaration to a foo(const char *) because the later declaration 
just promises more to you, and you lose nothing by foo()
making additional promises to you.
The takeaway is: you can pass a char * variable to a
function that wants a const char *. Both foo()
functions are accepting the same type. You can see
this by the compiler accepting the call to constp()
for a char * variable.

Thats different when a double pointer is involved as
with the constpp() function. Here you have to pass
a const char ** because otherwise the pointer value itself could
be assigned a char * by which const objects could be modified.
This would violate the const correctness of the program
and obsolete the const keyword. Thats why the foo(char **)
and foo(const char **) arguments are really different types.
In other words, you cant just add a const keyword to
double pointer function arguments as you could do it
with single pointers. You end up having entirely different
function signatures.

Now, guess what OpenSSL has made with the

DH_get0_key(const DH *dh, BIGNUM **pub_key, BIGNUM **priv_key);


RSA_get0_key(const RSA *r, BIGNUM **n, BIGNUM **e, BIGNUM **d);

functions? They "just" added a const to the BIGNUM
double pointers somewhere after the 1.1.0 version
already introduced a new API. Thats a warning for
C11 programs (one that you should not ignore) but even
worse, as you can see in above demo, its an error for
C++11 programs.

So, I had to add a wrapper function for the functions
in question which call the right functions, depending
on the OpenSSL version.

If you want to read more about the double pointer
const topic, its described here.

No comments: