Till Straumann <strauman@slac.stanford.edu>, 6/2012.
caxy is now also available as an implementation in java. This means that you don't have to compile anything but you can use the pre-compiled application on any platform supporting java (>= 1.5; 1.6 required for auto_address_list support).
The CA protocol uses both, TCP and UDP messages. While it is easy to tunnel the former using the powerful port-forwarding features of SSH it is not straightforward to tunnel UDP messages.
This document explains how to set up a TCP tunnel and documents the caxy proxy software that encapsulates (and modifies) CA-UDP messages for tunneling them via TCP.
On outside machine issue (in command line window):
java -jar caxy.jar -- ssh -D1080 user@insidehost java -jar caxy.jar -I Using PUTTY; we assume a complete configuration 'inside'
has already been created and saved under the name 'inside'. We also
assume authentication has been resolved either using GSSAPI or public key
files etc. As a last resort, you'd have to append a -pw
<password_in_clear> option to the plink command here:
java -jar caxy.jar -- plink inside java -jar caxy.jar -IOnce the tunnel is up, the message 'CAXY -- tunnel now established' appears. The tunnel remains running in the shell's 'foreground' and you can take it down by hitting <Ctrl-C> to kill it (and a few ugly error messages appear).
You have to tell CA clients to use the tunnel by
using a PATCHED EPICS CA library and setting:
export EPICS_SOCKS_PROXY=localhostproxify current session (e.g., for DANTE -- note that the config file has to be set up beforehand; consult DANTE docs)
. socksifyNow you can use CA clients normally:
caget my_PVThe most flexible way for tunneling CA TCP connections is using ssh's SOCKS proxy implementation together with a SOCKS wrapper implementation (a so-called proxifier) like tsocks or dante (client) or alternatively: patched versions of the EPICS CA client libraries (C/C++ or java, respectively). The patches which implement native SOCKS support for EPICS-CA have been submitted to the maintainers but are not yet present in the current releases of EPICS base or CAJ. Using these patches obsoletes the need for a proxifier.
Assume we have two hosts 'inside' and 'outside'. 'inside' is directly connected to the CA network, i.e., it can directly communicate with EPICS IOCs, receives their CA search replies, beacons etc.
The 'outside' host may only connect to the CA network via ssh.
On the 'outside' machine execute
ssh -D 1080 user@insideThis sets up a SOCKS proxy on the standard port 1080. The ssh proxy - in combination with the SOCKS-patch for EPICS or (alternatively) a so called 'proxifier' wrapper library - can transparently tunnel TCP connections. E.g., if you issue
tsocks telnet another_inside_machineThen you can transparently telnet over ssh and here's how it works:
Consult tsocks or dante documentation for how to set up those packages (it is really easy; just have to define routes that use the proxy instead of trying to establish direct connections).
A patch 'base-<vers>-socks-proxy.diff' to EPICS-base is
available which adds SOCKS support to EPICS. This means that you can execute CA
clients without the need for proxifier software (free versions of such software
seems hard to come by under windows). You also don't have to worry about
learning how to set up the proxifier.
The patch adds a new function 'epicsConnectViaProxy()' which is
a plug-in replacement for 'connect()' but knows how to negotiate
with a SOCKS proxy server when establishing a connection.
The patch also replaces the call to 'connect()' in the CA
client library with a call to epicsConnectViaProxy().
You can define a proxy by setting the environment variable
'EPICS_SOCKS_PROXY' to point to your proxy server. If you use an
ssh tunnel with 'dynamic port forwarding' then the ssh client
(running on your machine) is the SOCKS proxy server. Hence you just need to
export EPICS_SOCKS_PROXY=localhostand you're done. Easy enough, isn't it? If the
EPICS_SOCKS_PROXY variable is not defined then
epicsConnectViaProxy() behaves just like an ordinary
'connect()', i.e., a direct connection to the target is
established.
Note: If your CA libraries are dynamic libraries then you dont have to rebuild all your CA applications! Just patch EPICS base and rebuild base. Nothing in the API that would compromise binary compatibility is changed by this patch.
A analogous patch is available for the CAJ native java implementation of CA. Consult the section about CAJ and CSS below.
This is trickier. While SOCKS-5 does define support for the UDP protocol this is not implemented by ssh. Hence, tunneling CA-UDP does not work out of the box.
An alternative to the 'caxy' program would have been a more generic proxy that handles SOCKS-5/UDP and tunnels the UDP frames via TCP/SSH. However, SOCKS-5/UDP only supports UDP 'connections' that originate at the SOCKS client (i.e., frames being sent to a peer and the peer repeating back to the sender/client). Hence, even with SOCKS-5/UDP it would not be possible to relay CA beacons which originate at EPICS IOCs.
This is where the 'caxy' program comes in. This program wraps CA-UDP messages and ships them through a dedicated TCP tunnel (which is set up in the usual way with ssh).
The same program is executed (with different flags) on the 'inside' and 'outside' machines, respectively.
A UDP socket on port CA_SERVER_PORT (overrideable with the
EPICS_CA_SERVER_PORT env-var) listens for incoming CA packets. The
packet is then written to a TCP connection along with wrapping information that
identifies the sender of the packet.
Here is a diagram:
UDP communication ::
Machine-network boundary - - - - -
TCP communication ||
IPC boundary . . . . .
TCP encrypted XX
IPC between ssh/caxy (**)
UDP port |__|
TCP port ( )
CA-client
____
| | UDP port
- - - - - - - - - - - - - - - - - - - - - - - -
: ^
: :
CA_SEARCH 'outside'
: : network
request : : reply
V :
- - - - - - - - - - - - - - - - - - - - - - - - -
|____| UDP port 5064
outside :: (CA srvr port)
CAXY ::
encaps/decaps 'outside'
|| host
. . . . . . . . (**) . . . . . . .
SSH (clnt) ||
( )
- - - - - - - - - - - - - - - - - - - - - - - - -
XX
XX secure TCP Internet
XX Channel
- - - - - - - - - - - - - - - - - - - - - - - - -
( )
SSH (srvr) ||
. . . . . . . . (**) . . . . . . .
||
inside decaps/encaps 'inside'
CAXY :: host
::
| | UDP port
- - - - - - - - - - - - - - - - - - - - - - - -
: ^
: :
CA_SEARCH 'inside'
request : : reply (CA) network
V :
- - - - - - - - - - - - - - - - - - - - - - - - -
|___| UDP port 5064
CA server (IOC) EPICS IOC
receives wrapped UDP messages from TCP tunnel, unwraps and sends to all IOCs
listed in EPICS_CA_ADDR_LIST (as defined for caxy running
'inside').
For each 'sender' (in the outside world) the 'inside' caxy program maintains a UDP socket as a proxy so that replies arriving at that socket can be sent back (via the TCP tunnel) to the 'sender' of a request on the 'outside'.
When UDP messages arrive at a proxy socket then 'inside' retrieves the 'sender' address information from a trivial 'database', wraps the UDP message together with this info and stuffs it into the TCP tunnel.
receives encapsulated UDP message, unwraps, retrieves 'sender' address and sends back to 'sender'.
The CA protocol (since V48) support a feature where the IOC sending a search reply embeds its own address into the reply message. If this embedded address is undefined then the CA client assumes the sender of the reply is the IOC. Obviously, this would not work with this type of proxy (w/o resorting to tricks that intercept system calls). Therefore, caxy ensures that the server address is always embedded into search replies. Note that caxy does not work with very old (pre-V48) IOCs and/or clients.
In a simliar way, CA repeater traffic is proxied and tunneled. The 'inside'
caxy program registers with a caRepeater
that must be running on the 'inside' machine (caxy
does not attempt to launch a repeater).
Any traffic received from the 'inside' repeater is forwarded to the outside and
sent to the repeater port on the outside machine (if a caRepeater
is running on the outside machine then all subscribed clients will receive
beacons). Likewise, the `outside' caxy does not
attempt to start a repeater nor check if one is running.
CA-server
(IOC)
___
| | UDP port
- - - - - - - - - - - - - - - - - - - - - - - -
:
: 'inside'
CA-BEACON (CA) network
:
V
- - - - - - - - - - - - - - - - - - - - - - - -
|___| UDP port 5065
inside : (repeater port) 'inside'
REPEATER :: host
| |
. . . . . . . . . . . . . . . .
:^
beacon :: subscription 'loopback'
V: network
. . . . . . . . . . . . . . . .
|____|
:
inside :
CAXY encaps 'inside'
| host
. . . . . . . . .(**) . . . . . . .
SSH (srvr) |
( )
- - - - - - - - - - - - - - - - - - - - - - - - -
X
X secure TCP Internet
X Channel
- - - - - - - - - - - - - - - - - - - - - - - - -
( )
SSH (clnt) |
. . . . . . . . .(**) . . . . . . .
|
outside decaps 'outside'
CAXY : host
_:__
| | UDP port
. . . . . . . . . . . . . . . .
:
BEACON 'loopback'
: network
V
. . . . . . . . . . . . . . . . .
|___| UDP port 5065
outside 'outside'
REPEATER ::: host
. . . . . . . . ::: . . . . . . .
VVV Fanned-out beacons
- - - - - - - - - - - - - - - - - - - - - - - - -
CA Clients
For the latest information, check the output of caxy's '-h' option!
caxy supports the following options/switches:
EPICS_CA_ADDR_LIST env-var. Multiple -a options
and the env-var are combined into a single list. Only effective in
'inside' mode. EPICS_CA_AUTO_ADDR_LIST env-var or the 'auto_addr_list'
java property. However, this flag has the final say. It can be set to
'true' or 'false' but any value but 'true' is interpreted as 'false' (=
do NOT use the auto address list). Enable debugging messages. <flags> is a bit-set composed of
0x1: dump UDP messages
0x2: dump TCP messages
0x8: suppress beacon messages from debug output
0x10: trace lifecycle of threads
0x10000: dump raw tunneled message contents
0x20000: trace how java properties are looked up
0x40000: trace rehow CA addr list is assembled
The advantage of this approach is that the vanilla STDIO of ssh can be used w/o the extra need of a forwarded connection and the need for an open TCP port on the 'inside' machine which may collide with other users'. The only disadvantage is that it is not trivial to nail two pipes together which connect caxy to the ssh client. This is especially true under windows.
Luckily, caxy now helps you with this task: see '--'
NOTES: If you use this switch (with a non-zero value) then it indicates the TCP port caxy should connect to for tunneling its wrapped UDP traffic. SSH must forward this port to the 'inside' caxy. This switch ONLY applies to the TCP, not the UDP port. The TCP port MUST match the ports forwarded by ssh.
You probably don't need to use this option.
Accept connections from anywhere (only relevant on 'inside' and when not using STDIO for tunnel traffic; see -p, -S). By default only connections from the machine where caxy is running are accepted.
'server-mode'. Like '-I' but manage/accept multiple connections from multiple 'outside' clients. The main program remains running in the foreground. In this mode multiple (unrelated) 'outside' clients can create ssh tunnels from anywhere and connect to one single, shared caxy server. The caxy server may be started -- e.g., by a startup script -- locally on an inside machine.
It is strongly recommended to separate such a command with the option terminator: '--' so that any options given to the child command are not parsed by caxy itself but passed on to the child process.
Typically you would launch an 'ssh' session to the 'inside' machine where you would execute an 'inside' version of caxy:
java -jar caxy.jar -- ssh -D 1080 user@inside java -jar caxy.jar -Iwould bring up a complete tunnel.
Note that the stderr stream of the child process is copied (by a dedicated thread) to stderr of the JVM on the 'outside' machine so that you can see any potential error messages. A child process is only created by caxy running in 'outside' mode (it uses java's System.exec() method).
NOTE: Environment variables are ONLY read if the system property 'jca.use_env' is set to 'true' or unset and if the JCA property 'use_env' ('JCA' properties use one of the JCA prefixes, see '-J' above) is either not set or set to 'true'.
If 'use_env' is determined to be 'false' then environment variables are ignored and JCA properties 'server_port', 'repeater_port', 'addr_list' and 'auto_addr_list' are looked up, respectively.
JCA properties have the prefix as given (in order of precedence) by '-J <prefix>', or the string 'gov.aps.jca.Context' as a fallback.
Properties are first looked up in the user properties file (path itself defined by JCA property 'gov.aps.jca.JCALibrary.properties' or if such a property is not found then '.JCALibrary/JCALibrary.properties' in the user's home directory is used). If no user-specific property is found then the system-wide ones (located in 'lib/JCALibrary.properties' in the 'java.home' directory) are consulted and finally, a set of built-in resources 'JCALibrary.properties'.
This scheme essentially follows what JCA is doing. The features provided by '-J' as well as system and JCA properties are aimed at easing interoperability with JCA.
EPICS_CA_SERVER_PORT on the inside and outside.
In 'inside' mode: unless this is set to 'NO' a list of all broadcast addresses of the host's network interfaces is assembled and appended to the list built with -a options and EPICS_CA_ADDR_LIST.
This feature is only available under java 1.6 and higher.
The auto address list can also be forced on or off by means of the -A true / -A false command line option which takes precedence if present.
NOTE: The server and repeater port definitions may differ on the inside and outside. The tunnel handles this transparently but the settings must of course be correct. I.e., the values for caxy on the 'outside' must match the settings for CA clients and outside caRepeater and the values for caxy on the 'inside' must match the settings for CA servers and caRepeater on the 'inside'.
This feature is quite useful if the 'inside' CA network uses non-standard ports. It is only necessary to communicate these non-standard settings to caxy on the 'inside' (e.g, via a acript that sets the relevant environment variables or java property files). On the 'outside', caxy and the CA-clients do not have to replicate the non-standard settings but may just use the default ones.
# Start up a tunnel using STDIO to communicate with the inside: java -jar caxy.jar -- ssh -D1080 user@insidehost java -jar caxy.jar -I # Use PATCHED version of EPICS base; export EPICS_SOCKS_PROXY=localhost caget some_PV edm -x some_edm_screen
# Start up the tunnel as in the example above
java -jar caxy.jar -- ssh -D1080 user@insidehost java -jar caxy.jar -I
# from now on all programs are 'socksified'
. socksify
caget some_PV
# corresponding dante.conf routes (only 'from' lines):
# all tcp connections to a.b.c.0/24
from: 0.0.0.0/0 to: a.b.c.0/24 via: 127.0.0.1 port = 1080
...
# Assume: CA on a.b.c.255 subnet uses port 5066 # We use port 5056 for caxy's tunnel instead of STDIO # (assume inside has a script 'caxy' which starts up caxy.jar) ssh -t -D 1080 -L 5056:localhost:5056 user@insidehost caxy -I -p5056 -a a.b.c.255:5066 # caxy on outside is not socksified but CA clients are: java -jar caxy.jar -p 5056 & . socksify caget some_PV
# Same as previous example but caxy's tunnel uses the SOCKS proxy, too ssh -t -D 1080 user@insidehost java -jar caxy.jar -I -p5056 -a a.b.c.255:5066 # caxy on outside does use SOCKS; CA clients are proxified java -DsocksProxyHost=localhost caxy.jar -p 5056 & . socksify caget some_PV
# Note that anyone on 'hop' can possibly use the SOCKS proxy to 'inside'
# If this is a problem then a more sophisticated, encrypted tunnel
# must be set up. Also (and in the above examples alike), anyone
# on 'outside' can use the proxy port 1080.
java -jar caxy.jar -- \
ssh -L 1080:localhost:1080 hophost \
ssh -D 1080 insidehost \
java -jar caxy.jar -I -a a.b.c.255
# More importantly, this only works for ONE user on 'hop'. A second user
# would find 1080 already in use and would have to use a different
# port on 'hop', e.g., 3333:
java -jar caxy.jar -- \
ssh -L 1080:localhost:3333 hophost \
ssh -tt -D 3333 insidehost \
java -jar caxy.jar -I -a a.b.c.255
# use socksified CA client
. socksify
caget some_PV
# a better approach to set up a multihop tunnel is using ssh's
# 'ProxyCommand'. This avoids any intermediate port on the
# gateway/'hop' machine. It is more convenient to set this
# up in the 'config' file.
# The example uses command line options just to illustrate
# the setup:
java -jar caxy.jar -- \
ssh -D 1080 -o 'ProxyCommand=ssh hophost \
/usr/bin/nc %h %p' insidehost \
java -jar caxy.jar -I -a a.b.c.255
# Under windows, caxy works perfectly, e.g., in combination with the # PUTTY ssh client. # Especially for a multi-hop setup you would want to set up a configuration # (e.g., 'inside') which you then call up from the command line (plink). java -jar caxy.jar -- plink inside java -jar caxy.jar -I
It is possible to tunnel CA for client applications which use the 'CAJ' java implementation of the CA protocol such as the CSS suite.
The caxy tunnel must be started exactly in the same way as described above.
However, 'proxify-ing' the JVM may or may not work on different platforms. Therefore, we recommend a different approach:
Some network-related java classes already support connections via a SOCKS proxy. Hence, this built-in support can be leveraged. When setting the system property 'socksProxyHost' then all TCP connections established by 'Socket' objects are transparently going through the proxy.
Unfortunately, the 'SocketChannel' class which is used by CAJ (version 1.1.7) completely ignores this built-in support.
A patch to CAJ has been developed (and submitted to the CAJ maintainers, so it will hopefully be incorporated into future CAJ releases) which adds the necessary support for SOCKS. (Consult the 'ProxiedSocketChannel.java' file for more detail.) With this modified CAJ (and the tunnel set up as described in the previous sections) you could e.g., launch an application as follows:
java -DsocksProxyHost=localhost someCAJApp myPV
The patch to CAJ serves the same purpose (but targeting the java implementation of CA) as the 'base-<vers>-socks-proxy.diff' to the C/C++ version of EPICS base.
Eclipse also has built-in support for SOCKS proxies (you can define a proxy under Preferences-General-Network_Connections) but out of the box this does not interplay with (patched) CAJ or the java built-in SOCKS support in the Socket class.
Luckily it is just a matter of installing a small plugin to glue these pieces together. Hence, you need the following ingredients:
Now you are able to tunnel your CA connections:
It is worthwhile noting that the patched CAJ as well as eclipse with the 'org.eclipse.scout.net' plugin continue working normally when not told to use a proxy (e.g., when you plug your laptop into an 'inside' network). You only must remember to disable the proxy in the 'Network_Connection' settings and/or remove the 'socksProxyHost' system property.
If CAS is enabled and configured on an IOC then you may be denied access to a PV. Note that some CA clients produce no warning or error, the PV just seems to be unreachable or unwriteable.
To understand the issue here it is necessary to know that CAS authentication is based on a 'user name' and a 'host name'. These names are retrieved on the machine where the CA client is running and transmitted to the IOC 'inside' the CA protocol. Hence, it is irrelevant from which machine the CA/TCP connection originates (the IOC could also check that but it doesn't) which in our case would be the 'inside' machine (which probably is 'allowed' by CAS rules).
An example: When you are logged into your laptop 'mymachine' as 'Bob' then these credentials ('Bob/mymachine') are transferred by the CA protocol and will be checked by the IOC against its CAS rules.
Therefore, you must convince the IOC administrator(s) to add your name/machine to these rules.
'CATUN' is the (trivial) protocol caxy uses to wrap UDP datagrams for transmission over a stream. The first version of this tunnel protocol wrapped each individual CA message before shipping it through the stream/TCP connection. On the receiving end of the serialized stream, each CA message would be unwrapped and sent out in a UDP datagram. This means that even though the CA client might have packed multiple CA messages into a single UDP datagram, once this datagram was shipped through the tunnel (using the CATUN V1 protocol) it was broken up into multiple packets, each one holding only a single CA message.
CATUN protocol V2 remedies this and wraps an entire UDP datagram so that the original packet structure is preserved.
Note that the java version of caxy ONLY support protocol version 2. Hence it is not possible to co-operate with older versions of the C implementation.
CATUN protocol V3 defines a small albeit not backwards compatible addition to the initial handshake procedure which enables the outside to detect that the the tunnel has been established.
If you want to mess around with the source code or simply rebuild caxy (e.g., for running under java-1.5) then you need
Change directory to where 'build.xml' resides and issue
'ant'
The following targets are defined