This is adapted from the FlightAware fork, with some cleanup and
modifications needed to work with the net-cleanup changes.
Inclusion of "verbatim" TSV data read from an AVR-format input
connection is not supported.
realpath() returns a heap-allocated buffer if given NULL for the destination buffer.
This must be freed by the caller; dump1090 does not do this.
Instead of worrying about freeing it, take the simpler approach of just providing a
stack-allocated destination buffer.
Mostly refactoring the common code that was duplicated
between the different output types so that there aren't
many copies floating around.
This introduces "struct net_writer" to store the state of a
particular type of output service - buffers, time of last write,
connection count etc. prepareWrite() / completeWrite() give access
to the buffer and handle the actual writes and flushing when needed.
Heartbeat and time-based flushing move into a generic periodic-work
function.
Update the SBS output code to use the new infrastructure. This makes
a big different to CPU use when under load.
When we read from some client A, we may end up forwarding a message to other
clients. If we forward to some client B and there is a write error, then
we close B and remove it from the client list. However, if before this happened
A->next == B, then the read loop will still be holding on to a pointer to B,
and we crash.
As it's unpredictable what clients could be closed at what point, the simplest
approach is to retain closed clients in the list until we are at a point where
we know there are no stray pointers on stack, and only then modify the list.
This also simplifies anything that has to loop over clients, as it doesn't need
to worry about the current client being freed under it.
Client disconnection appears as a read of 0 bytes.
Without a test for this, dump1090 continues to poll that client forever.
Also, read() may return EWOULDBLOCK as well as EAGAIN
for "no data right now", so handle that.
I don't know if there is an equivalent Win32 bug here as the Win32
interfaces seem subtly different to vanilla POSIX.
The following test/break can probably be removed if Win32 needs
the same fix.
For remotely received messages that have a mlat timestamp, we have no
useful way of turning that timestamp into a wallclock timestamp, so
don't try, or we'll just produce wildly wrong results (_days_ in error)
It seems server code should be compatible with HTTP 1.1; the features
unique to 1.1 mostly are upon the client to support, and some headers
used (for example Cache-Control) may need 1.1.
A few minor additions and bug fixes as detailed below
1) Additional command line option "--net-buffer <n>" to specify the TCP
output buffer size. Default is n=0, which is 64Kb. Specify a value of n
to increase the buffer size according to Size = 64Kb * 2^n, so an n of
1 = 128Kb, n=2 is 256Kb etc. n is limited to 7, so the max size is 8Mb.
This option may assist if you have a high number of aircraft being
received, and an unreliable network connection, or if the receiving end
can be busy for an extended time.
2) Bug fix in ppup1090 which prevented the uploading of valid
ModeA/Squawk codes
3) Bug fix per Markus Grab's commit.
Make the modifications necessary to compile dump1090 for WinXP, Win7 and
hopefully Win8.
The files can be compiled using M$ Visual Studio/C++ 6.0. Due to various
licensing issues, I haven't included the libraries or DLLs. You will
need to locate pthreadVC2.lib and rtlsdr.lib to link the file, install
the zadig drivers to support the dongle, and locate libusb-1.0.dll,
msvcr100.dll, pthreadVC2.dll and rtlsdr.dll.
dump1090.exe will not run on any Windows version prior to XP SP2,
because msvcr100.dll imports several functions from the Windows kernel
that are not available on earlier versions. This means dump1090 won't
work on Win2K.
The major change to the code relates to file handles. The original code
assumes Linux behaviour in that handles are allocated from 0
sequentially upwards. However Windows handles are allocated pseudo
randomly, and handle numbers greater than 1024 would break the code. The
code has therefore been modified to use a linked list of connection
structures, rather than a static array limited to 1024 entries.
Thanks to Blort on the PP list.
The Beast binary message stream uses the 0x1a character a and escape to
mark the beginning of a new message. However, the 0x1a character could
occur in the body of a message since the message is binary. Therefore,
the 0x1a is repeated -as Blort put it :
This 56-bit Mode S Frame (containing a 1a) is supposed to look like
this:
1a 32 00 00 48 7b a6 1a 1a 0c 20 28 17 b0 c0 c3 b0
What is actually looks (looked) like (from wireshark) is this:
1a 32 00 00 48 7b a6 1a 0c 20 28 17 b0 c0 c3 b0
There are supposed to be two 1A’s (Gunter’s Escape Character) in a row
(1A 1A) whenever a data byte contains 1A, after the initial two 1A 3x
characters.
The Mutex on the RTL data reader thread does not "force" the data
processing thread to execute. Therefore, if the processor is busy, it is
possible for a second RTL callback to occur before the data from the
first has been processed. This will cause the loss of the first data,
but worse, it will cause a slip in the timestamp. This upsets Beamfinder
and MLAT operation in PlanePlotter.
To solve this, keep a Fifo buffer which is filled by the callback
thread, and emptied by the data processing thread. The fifo is the same
size as the number of buffers requested in the call to
rtlsdr_read_async().
Note - we only put the value of the pointer supplied in the callback
into the fifo. We do not attempt to cache the data in the buffer pointed
to by the pointer. This would require us to memcopy() 2Mbytes per
second, which we don't want to do if we don't have to because it will
only make the processor loading worse. Instead, we assume that the data
in the buffer will remain valid after the callback returns, at least
until it is overwritten by new data.
It is still possible for us to lose data if we can't process it quickly
enough. However, we can now detect this loss of data when the fifo is
almost full, and correct the timestamp for the lost block/blocks.
