I read two interesting articles on securing critical IT systems recently – rather diametrically opposed viewpoints on the role of the jump server or jump box as some people refer to it in securing the critical IT assets of an organization.
The first article was from 2013 by Roger Grimes discussing how to harden and secure your jump server. It’s a good read and provides some advice worth implementing.
The second article taking the contrarian view was by Rajat Bhargava and ran on O’Reilly’s TechRadar. In all fairness Rajat’s article discusses using jump servers in the cloud and argues they are less relevant – there are some merits to his arguments but I think he misses a few key points – one specifically highlighted in the notable Code Spaces hack or ‘corporate murder’.
Four Reasons I disagree with Mr. Bhargava… (although on most things I am very well aligned with his assertions!)
- I have not seen any mid-size or larger company that is 100% cloud for all IT assets. Do you have Switches, Routers, IP Phones, Firewalls, DNS/DHCP, etc? If so, you have assets on premises that need to be protected. As Sony, and Ashley Madison have both proved the insider threat should always be protected against.
- Password/Credential compartmentalization is incredibly powerful, ESPECIALLY in the cloud. Setup properly, and its not that hard, no admin that can see data, push code, or administer a system should ever have the actual password for that shared resource.
- Mr. Bhargava indicates that the jump box becomes a prime target. This is true. But the alternative is to have no defense-in-depth strategy that protects the core infrastructure from any compromised host. I would rather have a handful of tightly controlled and granularly logged hosts as my jump servers than put the management networks onto the same segments as easily compromisable users and hosts.
- ‘One Access Policy’ Myth – I really don’t know where the assertion that there is usually, generally, or often one access policy on a jump box comes from. Maybe if you are re-using an old Cisco 2514 and requiring someone to log-in to gain access to the admin side – but let’s face it if that is your security policy and implementation of a jump box you should really just post your SSH keys to Github and save the hackers a few minutes… a properly implemented jump box supports granular policies – and frankly operates very very well in a DevOps environment because security is integrated INTO your development framework and is not a bolt-on approach.
On the flip side of this Mr. Grimes wrote a piece highlighting steps you should be taking to secure jump servers – I was actually using Mr. Grimes guidelines as the basis for some automated audit reporting specifications the other day. A few that really jump out at me:
- Your jump server should never access the Internet, period. This is a wonderfully audit-able policy.
- The host that can access the jump server – it should be restricted to only SSL signed vendor websites. It should also only be rigidly segmented – VPN, VDI, or specific physical host depending on your security posture.
- All images should be signature signed and verified on the jump server.
- Policies should not be traditional permissive firewall – but should be hardened whitelists. Violations should trigger immediate alerts – indicating the administrator who was logged in at the time of the alert generation as well.
- Control Network Connectivity. Roger mentions VLANs which, because they are generally implemented and can be enforced on the switch are OK, but not great. Be sure to not have the switch port connecting to the jump server in wide-open 802.1q trunk mode. Be careful to avoid technologies like VXLAN for the management network though – without Source-VNI verification integrated into the network it is too easy to bypass the entire jump server and segmentation infrastructure.
Looking at many of the articles on jump servers, including Michael Ball’s espousing CyberArk, and reading up on Conjur there are some nice commercial offerings available. As anyone who has read our site knows we build a secure infrastructure platform – or as Russ Rice has educated me: cloud managed, on premises, secure server. A few things I would add to the list of things to put on your jump server checklist.
- Session recording. I don’t respect user privacy when on the machine that is administering my infrastructure.
- Anomaly monitoring. Block SCP, X11, look for large flows – could be covert data export.
- Credential compartmentalization. As indicated above, compartmentalize credentials so no one user credential lost or stolen can compromise networking equipment which has historically proven to have weak security policy and little/no image signing.
- Sign, Sign Sign… signed BIOS, OS, Images, Patches, and validate those regularly with Intel Trusted Execution Technology.
- Change passwords on the managed resources regularly. Lets say quarterly minimum, weekly may be overkill. But implemented right this does not require changing the admin multi-factor authentication.
In summary jump servers should be implemented between every management or OAM&P network and the network that the administrators reside on. A failure to do this is inviting trouble ranging from network/business outages to wholesale data loss. The insider threat is unfortunately quite real as Sony and Ashley Madison have learned – no single admin credential should ever be capable of destroying your business. As Mr. Bhargava accurately identifies this is a bit harder to do ‘in the cloud’ – but still quite possible with many of the more security focused cloud offerings available in the market today.