This lets different things dynamically create the services they need,
and sorts out the horrible hacks that view1090 used to make outgoing
connections. Now you can explicitly create a service and tell it to make
an outgoing connection.
This means that view1090 can now just set all the ports to zero (to disable
the listeners), do a normal net init, then explicitly construct the beast
input service without a listener and tell it to make a connection as needed.
This gives access to the existing throttling behaviour when reading
from a file with --interactive, without needing to actually have
--interactive.
This is useful when testing, as without --throttle samples will be
processsed much faster than real-time, which can produce different
results with e.g. aircraft/filter expiry times and position/speed checks.
This currently understands:
UC8 (original rtl_sdr format, 8-bit unsigned complex);
SC16 (16-bit signed complex, full width);
SC16Q11 (bladeRF native format: 16-bit signed complex with 11 bits of magnitude)
Magnitude conversion now happens immediately when sample data is
received, so there is no risk of newly received data clobbering old
data under CPU overload.
This is possible now that the SBS output doesn't rely on the global block timestamp;
the output will look like this:
MSG,8,111,11111,4AC954,111111,2015/02/08,17:57:53.917,2015/02/08,17:57:53.936,,,,,,,,,,,,0
MSG,7,111,11111,392AEB,111111,2015/02/08,17:57:53.744,2015/02/08,17:57:53.936,,15375,,,,,,,,,,0
MSG,8,111,11111,392AEB,111111,2015/02/08,17:57:53.917,2015/02/08,17:57:53.936,,,,,,,,,,,,0
MSG,6,111,11111,800387,111111,2015/02/08,17:57:53.919,2015/02/08,17:57:53.936,,,,,,,,4745,0,0,0,0
where the "receive timestamp" (first time column) goes backwards to reflect the original reception
time of the delayed message, but the "forwarded timestamp" (second time column) reflects the actual
forwarding time.
the message being emitted immediately.
Fix computation of reception time so it's more sensible (the block timestamp
is some time after reception of the _end_ of the block, not the start) - this
means that message-emission times are always later than message-reception
times in SBS output, which is a bit more sensible.
Use clock_gettime in preference to ftime.
(except in --net-verbatim mode, where we emit them all)
Move aircraft tracking into track.[ch].
Clean up references to "interactive mode" when tracking
aircraft - we always track aircraft, even in non-interactive
mode.
since we have 8 bits spare, so there's no chance of confusing it
with an ICAO address, and we can safely use the filter table to match
future messages without also matching equivalent ICAO addresses.
Switch signalLevel back to a power measurement, don't put SNR in there.
But make it a 0.0 - 1.0 double so we're not scaling everywhere.
Adjust for the amplitude offset when calculating power.
Adapt everything else to the new scheme.
This is adapted from the FlightAware fork, with some cleanup and
modifications needed to work with the net-cleanup changes.
Inclusion of "verbatim" TSV data read from an AVR-format input
connection is not supported.
Pull #54 breaks the Windows version of dump1090 due to a bug in the
inet_aton() windows function.
1) Fix the bug in the windows inet_aton()
2) Modify the command line code to strdup() the command line string.
3) Leave the Modes.net_bind_address pointer NULL if no command line
specified
Pull #54 permits changing the default bind address. The default was
127.0.0.1. This prevents external access unless a command line switch is
used. Since many users of dump1090 are using dump1090 as remote receiver
heads for programs such as plane plotter, this is undesirable. If you
want to lock down your RPi for local use only then use the command line
switch. If not, retain legacy open mode by binding to 0.0.0.0 by
default.
Mostly refactoring the common code that was duplicated
between the different output types so that there aren't
many copies floating around.
This introduces "struct net_writer" to store the state of a
particular type of output service - buffers, time of last write,
connection count etc. prepareWrite() / completeWrite() give access
to the buffer and handle the actual writes and flushing when needed.
Heartbeat and time-based flushing move into a generic periodic-work
function.
Update the SBS output code to use the new infrastructure. This makes
a big different to CPU use when under load.
When we read from some client A, we may end up forwarding a message to other
clients. If we forward to some client B and there is a write error, then
we close B and remove it from the client list. However, if before this happened
A->next == B, then the read loop will still be holding on to a pointer to B,
and we crash.
As it's unpredictable what clients could be closed at what point, the simplest
approach is to retain closed clients in the list until we are at a point where
we know there are no stray pointers on stack, and only then modify the list.
This also simplifies anything that has to loop over clients, as it doesn't need
to worry about the current client being freed under it.
If we demodulate a message in 2.4MHz mode and it has a bad, uncorrectable CRC,
and --phase-enhance is on, then retry with the other possible phases until
we get a good CRC or run out of phases to try.
This is very expensive in AGC mode (lots of candidates that are not real
messages) but relatively cheap otherwise. It yields another 10% messages.
Also factor out some common stats code to avoid lots more copy/paste.
There is a danger in always using relative decoding where possible.
If there is an undetected error in the first pair of messages received,
then global CPR decoding will give a bad position, and subsequent
relative decoding will just walk around near that bad position even
though many error-free pairs of odd/even messages may have been received.
The first pair of position messages also tends to be the most error-prone, as
they are usually received at the extreme edge of receiver range.
(I see this happen at least once a day in practice)
So, instead, prefer to use global decoding when we have sufficiently recent data.
With recent data this should always be as good as relative decoding, and it
avoids getting stuck with bad data for long periods of time. If we don't have
enough recent data for a global solution, fall back to relative decoding.
This disables most decoding of the contents of Mode S messages, aircraft tracking, and some output modes that depend on them.
It's intended for edge receivers that just forward to a central hub rather than processing data locally.
Release of COAA PlanePlotter MLAT and SMU support for RPi
ppup1090 now supports Ground Stations functions required for MLAT and
SMU operation. This is *ONLY* available for RPi and similar linux
hardware.
Also included are sample startup scripts for dump1090 only and
dump1090+ppup1090 together.
A few minor additions and bug fixes as detailed below
1) Additional command line option "--net-buffer <n>" to specify the TCP
output buffer size. Default is n=0, which is 64Kb. Specify a value of n
to increase the buffer size according to Size = 64Kb * 2^n, so an n of
1 = 128Kb, n=2 is 256Kb etc. n is limited to 7, so the max size is 8Mb.
This option may assist if you have a high number of aircraft being
received, and an unreliable network connection, or if the receiving end
can be busy for an extended time.
2) Bug fix in ppup1090 which prevented the uploading of valid
ModeA/Squawk codes
3) Bug fix per Markus Grab's commit.
Make the modifications necessary to compile dump1090 for WinXP, Win7 and
hopefully Win8.
The files can be compiled using M$ Visual Studio/C++ 6.0. Due to various
licensing issues, I haven't included the libraries or DLLs. You will
need to locate pthreadVC2.lib and rtlsdr.lib to link the file, install
the zadig drivers to support the dongle, and locate libusb-1.0.dll,
msvcr100.dll, pthreadVC2.dll and rtlsdr.dll.
dump1090.exe will not run on any Windows version prior to XP SP2,
because msvcr100.dll imports several functions from the Windows kernel
that are not available on earlier versions. This means dump1090 won't
work on Win2K.
The major change to the code relates to file handles. The original code
assumes Linux behaviour in that handles are allocated from 0
sequentially upwards. However Windows handles are allocated pseudo
randomly, and handle numbers greater than 1024 would break the code. The
code has therefore been modified to use a linked list of connection
structures, rather than a static array limited to 1024 entries.
Some users have reported issues where the TCP link to dump1090 can be
lost at times of low traffic density - typically in the middle of the
night. One possible reason for this is that some routers drop the link
if there is no traffic for a predetermined period.
To try and resolve this, dump1090 now sends a 'null' packet consisting
of 7 "0x00" bytes approximately once a minute if there is no real
received traffic during this time. This packet should be discarded by
the application receiving the dump1090 because it will have an invalid
checksum, and ICAO address 0x000000 is also invalid. However, this null
packet should be enough to keep routers alive.
The Mutex on the RTL data reader thread does not "force" the data
processing thread to execute. Therefore, if the processor is busy, it is
possible for a second RTL callback to occur before the data from the
first has been processed. This will cause the loss of the first data,
but worse, it will cause a slip in the timestamp. This upsets Beamfinder
and MLAT operation in PlanePlotter.
To solve this, keep a Fifo buffer which is filled by the callback
thread, and emptied by the data processing thread. The fifo is the same
size as the number of buffers requested in the call to
rtlsdr_read_async().
Note - we only put the value of the pointer supplied in the callback
into the fifo. We do not attempt to cache the data in the buffer pointed
to by the pointer. This would require us to memcopy() 2Mbytes per
second, which we don't want to do if we don't have to because it will
only make the processor loading worse. Instead, we assume that the data
in the buffer will remain valid after the callback returns, at least
until it is overwritten by new data.
It is still possible for us to lose data if we can't process it quickly
enough. However, we can now detect this loss of data when the fifo is
almost full, and correct the timestamp for the lost block/blocks.
Create separate makefiles for dump1090, view1090 and ppup1090. These can
be run with the command line "make -f makedump1090", "make -f
makeview1019" and "make -f makeppup1090"
Pass dump1090 version number into ppup1090 uploader so that coaa1090.obj
doesn't need re-compiling between versions.
Remove ppup1090 from general Makefile - it's not intended for anything
other than RPi, so gives linker errors on other (non Linux raspian)
systems.
Updated the way socket handles are used in View1090 to maintain
compatibility between UNIX and Windows.
Added the initial attempt at a Planeplotter uploader
No changes to dump1090, (except the version number)
Include a sample Linux batch start file called dump1090.sh for use when
running dump1090 headless. This file needs to be copied to the
/etc/init.d/ subdirectory on your raspberry pi, and marked as
executable. Then when you re-start your RPi, dump1090 will start-up
auto-magically and run as a sort of server to allow both local and
remote connection to it's various internet ports.
Modified the Makefile to build a new headless helper application called
view1090
Added view1090. This is an executable that allows you to connect to
dump1090 when it is running and 'see' the interactive screen display.
The default is to try and connect to dump1090 on IP address 127.0.0.1
port 30005. This should work if you are running on the same RPi as
dump1090 and using the default dump1090 port settings. However, if
you're running on a different machine you will have to specify the IP
address of the RPi running dump1090 using the --net-bo-ipaddr switch.
Something like "view1090 --net-bo-ipaddr 192.168.2.65" . You may also
have to sudo it, depending on your privilige settings.
I've also compiled view1090 as a Wiin32 exe, so you should be able to
run it under any 32 bit version of Microsoft Windows - i.e. Win95, Win
2K, Win XP, Win 7 etc. It may work on Win 8 and 64 bit Windows, but I
haven't tried it. The Win32 version is compiled from the same source, so
takes all the same command line switches.
The modesReadFromClient() funtion is called from modesReadFromClients(),
which in turn is called from backgroundTasks(). backgroundTasks() is
called from within the main processing loop.
However, modesReadFromClient() can and does block. It attempts to read
characters from the input stream, and loops whilst there was no error.
This stalls the main RTL processing loop until an error occurs. In order
to support simultaneous local reception (via our RTL dongle) and remote
forwarding (data received from the interweb) we cannot allow this
internet read to stall.
To fix this, in modesReadFromClient() attempt to read a buffer of data
(currently 0x400 bytes). If we get a full buffer of bytes, then process
them, and attempt to read another full buffer. Keep doing thios untill
we read only a partial buffer (less than 0x400 bytes). Process the
partial buffer bytes and return.
This allows us to occasionally process data that is arriving from the
internet (which is buffered anyway in the TCP stack), without blocking
local RTL dongle decoding.
Ok - this is likely to upset some people. Up until now, the vast
majority of the code has been in just one file - dump1090.c. This file
has grown so that it was approaching of 5000 lines long, and it was
becoming unmanagable. So I've split the file into several modules,
hopefully along fairly logical boundaries. The files are :
1) dump1090.c : Basically just the main() entry function, the help
function, the RTL dongle hardware interface, and a few orphan functions
that don't really fit anywhere else.
2) mode_s.c : This contains all the mode S / ADSB decoding functions.
3) mode_ac.c : This contains all the mode A & C decoding functions
4) interactive.c : This contains all the functions to maintain an
internal list of aircraft seen over the last period, and functions to
print them out to the local console.
5) net_io.c : This contains all the network input/output functions
allowing data to be passed in/out to/from other receivers, in formats
such as SBS-1/3, Beast, AVR and JavaScript.
Hopefully this should provide an easier way forward if/when more
functions are added.
