Compiling SoftEther VPN on a modern environment
This all started with a very simple need to run SoftEther VPN on a Docker container as per their own instructions... What I didn't realize was that such a simple task would require forking the project and patching makefiles...
Apart from Tailscale, which is great at creating a mesh network of all my modern devices, my go to VPN has always been SoftEther, it's a great choice if you want to quickly setup OpenVPN or L2TP with ease, apart from this, if you are like me and want to VPN your retro hardware, it's the best VPN software available.
I currently have 2 SoftEther servers running on the infrastructure that I manage and noticying that they had a Docker container available, I decided to try it out, since it may make setup, migration and backup of the server a lot easier in the future.
The first issue I noted was that their stable image was only compiled for Intel, not ARM64, which is usually what my cloud servers run. The new version of SoftEther, version 5, was available for ARM64, but due to incompatibilities with the old stable version and my reluctance in upgrading, I wasn't ready to try it out.
In order to have the stable version working under Docker on one of my ARM64
servers, I decided that it was easy enough to just grab
their
Dockerfile for the stable version and create a simple
docker-compose.yml to build that container image locally instead
of having to rely on the pre-built one from the Docker Registry. While I was at
it I also took care to change the Git tag in the Dockerfile since
that was pointing to an already years old version of the software, pointing it
now to the latest stable release.
Everything was going way too smoothly, and soon enough I quickly ran into an issue that will become very common with old C code bases as C23 support becomes mainstream and the default option in most C compilers:
> [builder 5/5] RUN cd SoftEtherVPN_Stable && git submodule init && git submodule update && ./configure && make:
0.715 In file included from ./src/Mayaqua/Mayaqua.h:334:
0.715 ./src/Mayaqua/MayaType.h:262:33: error: 'bool' cannot be defined via 'typedef'
0.715 262 | typedef unsigned int bool;
0.715 | ^~~~
0.715 ./src/Mayaqua/MayaType.h:262:33: note: 'bool' is a keyword with '-std=c23' onwards
0.715 ./src/Mayaqua/MayaType.h:262:1: warning: useless type name in empty declaration
0.715 262 | typedef unsigned int bool;
0.715 | ^~~~~~~
0.771 make: *** [Makefile:111: tmp/objs/Mayaqua/Cfg.o] Error 1The developers of SoftEther, due to its Win32 origins, decided to ensure they
had a compatible boolean type under UNIX, as they may be abusing the size of a
boolean in their code base, since on Windows it's defined as an unsigned
int and developers are used to taking advantage of its size sometimes.
The issue with this (re)definition is that in C23
bool
is now a proper keyword and thus you cannot typedef something
to it, just like you cannot typedef to an if or a
while.
First I tried to add a C standard directive to instruct the compiler to use an
older standard, just like
the good folks over at NixOS did. In order to do this I decided to add
suffixes to the make command, like so:
RUN cd SoftEtherVPN_Stable &&\
git submodule init &&\
git submodule update &&\
./configure &&\
make CFLAGS='-std=gnu17' CXXFLAGS='-std=gnu17'This did not work and produced the exact same errors as above, meaning my
directive was being completely ignored, specially when looking at
make's output, my flags weren't appearing in the invoked
compilation command.
I decided to look through the SoftEther stable repository pull requests to see if anyone had already encountered this issue before and contributed something that hadn't yet been merged. To my surprise, the developers may be completely ignoring the stable branch of SoftEther in favor of the new version 5 project, since there were some very trivial open pull requests that have been lingering unmerged for an entire year [1,2], although none of these had to do with my C23 issue.
I decided that enough was enough and it was time to maintain my own fork, with the changes they hadn't merged and the C23 fix. This would ensure that I could, in the short term, have my VPN server running inside Docker in one of my ARM64 cloud servers, at the cost of having to maintain a fork, which given the stability of their code base means doing nothing at all, so no issues on that front.
In order to ensure that my workaround for GCC version 14 and above, when C23 became the standard, was only applied when a modern version of GCC was used, thus abiding by their guidelines that the project must support GCC 2.x, in the makefiles I guarded the direction to add the C standard flag only when GCC version 14 and above was used:
ifeq "$(shell expr `gcc -dumpversion | cut -f1 -d.` \>= 14)" "1"
OPTIONS_COMPILE += -std=gnu17
endif