Deploy360 Domain Name System Security Extensions (DNSSEC) Events Improving Technical Security

RIPE 72 – Highlights from Day 4 & 5

RIPE 72The RIPE 72 meeting was happening last week in Copenhagen, Denmark, and we’ve been highlighting the presentations and activities related to the Deploy360 technologies. In this post we’re summarising the last couple of days of activity before the meeting drew to a close.

The Thursday was also mostly devoted to Working Groups, including the second part of the IPv6 Working Group. We have highlighted this before, but it’s worth mentioning again the report on IPv6 deployment in Latin America and the Caribbean from LACNIC. This is a comprehensive survey of the current state of IPv4 exhaustion, case studies of IPv6 deployments, but perhaps more interestingly the problems, challenges and regulatory barriers that were experienced. The report also provides insight into the rationale of those ISPs not currently considering the deployment of IPv6.

This was followed by a useful presentation on Going v6-only at home by Luuk Hendriks (University of Twente) which aimed to get an IPv6 only Wireless LAN up and running in a home environment. The rationale was to find a solution that is generally deployable and therefore not to have to buy any special hardware or require hacks, as well as discover which elements had shortcomings. It transpired that problems were mostly related to applications insufficiently supporting IPv6, but the primary issue proved to be a lack of AAAA records that effectively disabled IPv6. Whilst there are workarounds such as NAT64 and DNS64, there remains a question mark over how these can support DNSSEC as we highlighted last week.

On a similar theme was the presentation on How to Make Trouble for Yourself – You Build an IPv6-Only Network in 2016 from Roger Jørgensen (nLogic). This was about the challenges of building a next generation network serving the Norwegian county of Troms which is situated in the Arctic and requires the deployment of IPv6 to ensure it can support future requirements for the next 25-30 years. Given the remoteness of the region it’s essential that Customer Premises Equipment (CPE) is reconfigurable and IPv6 manageable, but the first challenge was a lack of DHCPv6 support which meant static IPv6 addresses needed to be configured via an uploadable config file which allows the CPEs to then be registered with the Junos Space Management Platform which does fully support IPv6.

The IPv6 Working Group session was rounded off with the results of a survey about which open source code repositories were available via IPv6. As of the end of 2015 there were 25,522 ports located on 5,344 hosts in the FreeBSD ports collection. Of these, 3,925 hosts only had an IPv4 address whilst not one host only had an IPv6 address, meaning there were 1,419 hosts with both an IPv4 and IPv6 address although 359 of these did not have a resolvable DNS record. Just 10,308 ports could therefore be downloaded with IPv6, although if you add in dependencies then this rises to 17,715 ports (69.4% of the total) that cannot be built by an IPv6-only user.

It’s worth balancing this up though, with the presentation from Jen Linkova (Google) during the final plenary session about Google’s public DNS64 server that is currently in beta. More information is available at

Over in the DNS Working Group there were several presentations related to DNSSEC. First up was Duane Wessels (Verisign) who summarised the changes to the Zone Signing Keys (ZSKs) for the root zone that is planned to change to 2,048 bits, followed by rollover of the Key Signing Key (KSK). This was followed by presentation on ZSK controlling from Paul Ebersman (Comcast), with the session rounded off with a panel discussion on the issues to address with the DNS. The DNS Manifesto summarises much of this, including the issues related to DNSSEC and the lack of data encryption.

Last, but not least we need to highlight a couple of appeals made during the Routing Working Group related to Deploy360 technologies. The first was on RPKI Validation from Alex Band (RIPE NCC) which summarised RPKI update in the RIPE region, as well provided a review of existing RPKI validation software including the RIPE NCC implementation. The RIPE NCC validator had been built somewhat as a proof of concept, but suffered from high CPU and memory usage with the growing data set, and did not leverage data from the Internet Routing Registry. They therefore wish to redesign this, and are asking for community feedback on the feature set.

The second appeal was made by Job Snijders (NTT) who asked network operators not to accept route announcements from external neighbours containing ‘bogon’ ASNs. NTT will adopt the policy of not accepting ASNs 23456, 64496-131071 and 4200000000-4294967295 from July 2016 as these numbers are variously reserved for private use, documentation/code examples or other reasons and should not be routed. They are normally seen due to misconfigurations or bugs, but can also be used maliciously and should be eliminated from the global routing table.

So that’s it from us for RIPE 72 which proved to be a week that broke all attendance records. All the presentations from the meeting can be found on the RIPE 72 website.

The next RIPE meeting will held on 24-28 October 2016 in Madrid, Spain. We look forward to seeing you then!


Deploy360 IPv6

RIPE 72 – Highlights from Day 3

RIPE 72The RIPE 72 meeting is happening this week in Copenhagen, Denmark, and we’re highlighting the presentations and activities related to the Deploy360 technologies throughout the week.

Wednesday was mostly devoted to Working Groups, and there are several interesting presentations to highlight. Most of the discussion in the Address Policy Working Group was about policies for managing the remaining IPv4 address blocks, which offers another indication that IPv4 addresses are reaching depletion and obtaining more is starting to attract significant cost. Network operators therefore really must have deployment of IPv6 at the forefront of their thinking.

The IPv6 Working Group was the focus of the day though, and once again featured a thought provoking presentation on Large Packets in IPv6 from Geoff Huston (APNIC). Packet fragmentation is allowed where a router cannot forward a packet due to a size mismatch, but this causes significant problems in the Internet and nearly 50% of fragmented IPv6 packets are lost. This then requires the entire packet to be re-sent, whilst fragments represent a security vulnerability as they are easily spoofed and also consume resources at the destination.

IPv6 capable links are supposed to offer a minimum MTU size of 1280 bytes, but this appears to encourage fragmentation, whilst increasing the MTU size encounters risks with MTU mismatch and the possibility that fragments are lost. Experiments undertaken with DNS queries over IPv6 clearly show this, and would appear to account for the number of unresolved IPv6 addresses that have been observed. This might suggest the need to revisit the deprecation of fragmentation support in IPv6.

We’ve mentioned this before, but John Jason Brzozowski (Comcast) provided an update on the Internet Draft draft-ietf-v6ops-unique-ipv6-prefix-per-host-00 which advocates the use of unique IPv6 prefixes for hosts. A community Wi-Fi service allows hosts to connect to a shared network providing Internet and/or other services, and in a typical network they’d acquire unique IPv6 addresses from a common IPv6 prefix through mechanisms such as SLAAC or DHCPv6. However, this can introduce some performance issues related to router and neighbour discovery, as well as security issues with numerous untrusted devices belonging to lots of different customers. By allowing allowing hosts to be assigned a unique IPv6 prefix (typically a /64), this would provide each subscriber with more flexibility to utilise IPv6, whilst ensuring traffic can be directed to a default wireless LAN gateway.

Another interesting issue was raised by Jen Linkova (Google) on using DNSSEC with IPv6-only networks. The problem on such networks is that the host does not have an IPv4 address to initiate a query to an IPv4-only DNS server, and whilst NAT64 can facilitate IPv6 to IPv4 communication, if DNS64 is needed to synthesise A records into AAAA records, then DNSSEC will fail. It is therefore important for DNS operators to ensure their servers support IPv6 and AAAA records, but that DNSSEC-aware and validating stub resolvers also to be DNS64 aware and able to discover the NAT64 prefix.

Following on from this, it’s worth checking out the RIPE Atlas measurement updates on IPv6 and DNSSEC-aware resolver deployments.

The Connect Working Group also had something of interest, with a presentation about RPKI Validation at IXPs from Daniel Kopp (DE-CIX). The aim was to promote acceptance and usage of cryptographic validation of IP prefixes in order to increase the security of the routing system by reducing prefix hijacking and route leaks. RPKI deployment was currently running at just over 8% of advertised address space, and was largely reflected at DE-CIX where 7.35% of prefixes could be validated which represented 12.78% of data volume. Conversely 0.75% of prefixes were found to be invalid.

There is an Internet Draft proposed by DE-CIX, AMS-IX, France IX and others to the IETF Secure Inter-Domain Routing (SIDR) Working Group that would allow RPKI validation results to be signalled from a route server to peers. This aim is to support legacy hardware and therefore encourage take-up by customers.

For those of you who cannot attend the RIPE meeting in person, remote participation is available with audio and video streaming and also a jabber chat room.

The full programme can be found at

Deploy360 Internet of Things (IoT) IPv6

RIPE 72 – Highlights from Day 2

RIPE 72The RIPE 72 meeting is happening this week in Copenhagen, Denmark, and we’re highlighting the presentations and activities related to the Deploy360 technologies throughout the week.

Tuesday was all plenary sessions, but to kick-off we must highlight the presentation on the Internet of Things: What is the problem? by Shane Kerr (Beijing Internet Institute). The Internet of Things (IoT) promises to be radically different to other technological developments on the Internet, and in particular the question needs to be answered about whether the Internet can be scaled to accommodate an exponential increase in the number of connected devices. Whilst the IPv6 address space should be sufficient for this and the DNS is hierarchical by design and has proved scalable so far, routing protocols are not designed in this way and potentially suffer from the problem of maintaining a large number of routes.

The big challenges with IoT though, are development of proprietary protocols, the unattended nature of devices, and the problems of built-in obsolescence. Consumers not only expect day-to-day appliances to be cheap, but also to be able to use them for many years. However, once a vendor sells a consumer product, then there’s very little incentive to maintain that product for very long given the low selling price. This can create significant problems if software is not being updated in response to security issues that are discovered.

One solution might be to have standardised open source operating systems for such devices, but again what is the economic incentive for vendors to move in this direction? It may therefore require industry agreement or regulatory intervention to ensure ongoing maintenance programmes, even assuming users are sufficiently trusting of IoT devices to replace their existing non-connected devices that may have already served them for many years. So whilst there’s an assumption that IoT is a substantial business opportunity, existing economic models do not currently fit well with vendor and consumer expectations and will require a rethink of standards and how products are supported in the long term.

Next up was the presentation on IPv6 @ Comcast by John Jason Brzozowski (Comcast). Comcast has been one of the leading ISPs for IPv6 deployment, with their IPv6 programme having started back in 2005. As of 2016, 98% of their devices are now managed using IPv6 only and is expected to reach 100% in the near future, whilst 25% of their Internet facing communication is using IPv6.

This demonstrates that it is possible to deploy native IPv6 without transitional technologies, and nearly 60% of Comcast traffic is now IPv6. The key has been to incrementally move particular segments of the network and services to IPv6, whilst having well formed plans in place to identify and fix issues as they’re encountered.

Last but not least are some interesting measurements from Vaibhav Bajpai (Jacobs University Bremen) on how websites compare on dual-stacked hosts. The Alexa top 100 websites supporting IPv4 and IPv6 were interrogated using simweb that compares the different results returned using each protocol.

The measurements showed that 27% of the websites showed some missing elements when using IPv6, 9% exhibited a greater than 50% failure rate using IPv6, whilst 6% failed completely. The failing elements tended to be attributable to DNS resolution errors on images, Javascript and Cascading Style Sheets, and often occurred on pages underneath the web root, so this is something that system administrators need to pay additional attention to.

For those of you who cannot attend the RIPE meeting in person, remote participation is available with audio and video streaming and also a jabber chat room.

The full programme can be found at

Deploy360 Domain Name System Security Extensions (DNSSEC)

RIPE 72 – Highlights from Day 1

RIPE 72The RIPE 72 meeting is happening this week in Copenhagen, Denmark, and looks to be the biggest ever with nearly 800 registered participants. Jan and Kevin from the Deploy360 team are here, and we’ll be highlighting all the relevant presentations and activities.

There were three interesting DNS presentations during the opening plenary session that are worth highlighting. The first by Paul Ebersman (Comcast) asked the question “What’s so hard about DNSSEC?” and related his deployment experiences. The main reasons to deploy DNSSEC are the ability to assert responses to DNS queries, to help protect against cache poisoning, and as a way of enabling DANE and other PKIs.

The oft-heard reasons for not deploying DNSSEC are that it requires a lot of additional work, tends to break things, and is reliant on ICANN and the root servers. The reality though, is that the DNS already relies on the root servers, customers increasingly expect improved security, and that DANE is already being used for validating non-web servers. Deployment of DNSSEC can require additional effort, but this can be substantially reduced through automation.

Whilst setting-up the signing of a zone requires some effort and key rollovers need to be handled carefully, Comcast’s experience was that their signed zones typically returned less than two dozen errors per month, whilst the need for Negative Trust Anchors (NTAs) was limited to single digits per month. The message therefore, is that using DNSSEC is not without problems, but most of these can be mitigated by careful planning and communication and has been substantially less painful than anticipated.

The second presentation was an alert from Patrik Fältström (Netnod) that Web Proxy Auto‐Discovery (WPAD) queries intended for resolution on private or enterprise DNS servers have been observed reaching public DNS servers. This could result in domain name collisions with internal network naming schemes, allowing these to be abused by configuration of an external proxy for network traffic that provides the potential for man‐in-the-middle attacks. This issue has been known about for at least three years, but the problem has been increasing as more second-level domains have been created within new gTLDs and more platforms have been found to be affected.

Network administrators are therefore being asked to disable automatic proxy discovery/configuration in browsers and operating systems, use fully qualified domain names (FQDNs), configure internal DNS servers to respond authoritatively to internal TLD queries, whilst configuring firewalls and proxies to log and block outbound requests for wpad.dat.

The third presentation was an analysis by Geoff Huston (APNIC) on DNS zombies – queries that have no user awaiting the response and are instead are echoes of previous queries. Over a five month experiment and detailed analysis of around 44 billion DNS queries, it was discovered that around 20%  these queries were zombies, but question was whether this behaviour is due to DNS resolvers continually sending out queries for which responses are never accepted, or whether this was due to something more sinister? As it transpires, it would seem to be attributable to misconfigured resolvers, of which just 11 are responsible for 60% of all zombie queries. This is explained in more detail in this APNIC article.

We’d also like to highlight the presentation from our Internet Society colleague Phil Roberts on Cryptech. This an open source hardware security module (HSM) reference design that can store, manage and process digital keys as used in DNSSEC, PKI, RPKI and PGP amongst other applications. The aim is to address concerns about potentially compromised devices in critical Internet infrastructure, and to ensure IETF protocols are supported in an open and transparent manner.

The first alpha boards are now working and will be available for external testing this summer, initially supporting DNSSEC with RPKI coming shortly afterwards. Cryptech was therefore looking for people with implementation and operational experience of these technologies to help undertake additional testing of these devices, as well as help with the documentation. More information is available on the Cryptech website.

Although not directly related to Deploy360, it’s also worth checking out a couple of presentations on the State of African Peering by Andrew Owens (Teraco) and Interconnection in the Nordics by Lasse Jarlskov (TDC) which provide a good overview on where and how peering happens in two distinct regions of the world.

Following the plenary session, Jan Žorž chaired the BCOP Task Force. There were two new BCOPs up for discussion relating to DNS Operations and Using your last IPv4 address, as well as an update on the MANRS (Mutually Assured Norms for Routing Security) BCOP. The meeting was also updated on BCOP developments in Africa and Latin America, some of which were discussed in our previous post on the AfBCOP Workshop.

For those of you who cannot attend the RIPE meeting in person, just a reminder that remote participation is available with audio and video streaming and also a jabber chat room.

The full programme can be found at