draft-arkko-arch-low-latency-00.txt		draft-arkko-arch-low-latency-01.txt
	Internet Engineering Task Force J. Arkko		Internet Engineering Task Force J. Arkko
	Internet-Draft Ericsson		Internet-Draft Ericsson
	Intended status: Informational May 15, 2017		Intended status: Informational J. Tantsura
	Expires: November 16, 2017		Expires: January 3, 2018 Individual
			July 2, 2017
	Low Latency Applications and the Internet Architecture		Low Latency Applications and the Internet Architecture
	draft-arkko-arch-low-latency-00		draft-arkko-arch-low-latency-01
	Abstract		Abstract
	Some recent Internet technology developments relate to improvements		Some recent Internet technology developments relate to improvements
	in communications latency. For instance, improvements in radio		in communications latency. For instance, improvements in radio
	communications or the recent work in IETF transport, security, and		communications or the recent work in IETF transport, security, and
	web protocols. There are also potential applications where latency		web protocols. There are also potential applications where latency
	is potentially in a more significant role than it has traditionally		is potentially in a more significant role than it has traditionally
	been in Internet communications. Modern networking systems offer		been in Internet communications. Modern networking systems offer
	many tools for building low-latency networks, from highly optimised		many tools for building low-latency networks, from highly optimised
	skipping to change at page 1, line 40		skipping to change at page 1, line 41
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF). Note that other groups may also distribute		Task Force (IETF). Note that other groups may also distribute
	working documents as Internet-Drafts. The list of current Internet-		working documents as Internet-Drafts. The list of current Internet-
	Drafts is at http://datatracker.ietf.org/drafts/current/.		Drafts is at http://datatracker.ietf.org/drafts/current/.
	Internet-Drafts are draft documents valid for a maximum of six months		Internet-Drafts are draft documents valid for a maximum of six months
	and may be updated, replaced, or obsoleted by other documents at any		and may be updated, replaced, or obsoleted by other documents at any
	time. It is inappropriate to use Internet-Drafts as reference		time. It is inappropriate to use Internet-Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	This Internet-Draft will expire on November 16, 2017.		This Internet-Draft will expire on January 3, 2018.
	Copyright Notice		Copyright Notice
	Copyright (c) 2017 IETF Trust and the persons identified as the		Copyright (c) 2017 IETF Trust and the persons identified as the
	document authors. All rights reserved.		document authors. All rights reserved.
	This document is subject to BCP 78 and the IETF Trust's Legal		This document is subject to BCP 78 and the IETF Trust's Legal
	Provisions Relating to IETF Documents		Provisions Relating to IETF Documents
	(http://trustee.ietf.org/license-info) in effect on the date of		(http://trustee.ietf.org/license-info) in effect on the date of
	publication of this document. Please review these documents		publication of this document. Please review these documents
	skipping to change at page 2, line 16		skipping to change at page 2, line 17
	to this document. Code Components extracted from this document must		to this document. Code Components extracted from this document must
	include Simplified BSD License text as described in Section 4.e of		include Simplified BSD License text as described in Section 4.e of
	the Trust Legal Provisions and are provided without warranty as		the Trust Legal Provisions and are provided without warranty as
	described in the Simplified BSD License.		described in the Simplified BSD License.
	Table of Contents		Table of Contents
	1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2		1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
	2. Applications with Special Focus on Low Latency . . . . . . . 3		2. Applications with Special Focus on Low Latency . . . . . . . 3
	3. Role of Low-Latency vs. Other Communications . . . . . . . . 4		3. Role of Low-Latency vs. Other Communications . . . . . . . . 4
	4. Selected Improvements to Communications Latency . . . . . . . 4		4. Selected Improvements to Communications Latency . . . . . . . 5
	5. Architectural Considerations . . . . . . . . . . . . . . . . 5		5. Architectural Considerations . . . . . . . . . . . . . . . . 5
	5.1. Background . . . . . . . . . . . . . . . . . . . . . . . 5		5.1. Background . . . . . . . . . . . . . . . . . . . . . . . 6
	5.2. Implications . . . . . . . . . . . . . . . . . . . . . . 6		5.2. Implications . . . . . . . . . . . . . . . . . . . . . . 7
	5.3. Recommendations for Further Work . . . . . . . . . . . . 8		5.2.1. Service Distribution . . . . . . . . . . . . . . . . 7
	6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 8		5.2.2. Edge Computing . . . . . . . . . . . . . . . . . . . 8
	7. Informative References . . . . . . . . . . . . . . . . . . . 9		5.2.3. Routing and tunnels . . . . . . . . . . . . . . . . . 8
	Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 11		5.2.4. Alternative Paths and Control Tension . . . . . . . . 8
			5.2.5. Cross-Layer Optimisations . . . . . . . . . . . . . . 9
			5.3. Recommendations for Further Work . . . . . . . . . . . . 10
			6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 11
			7. Informative References . . . . . . . . . . . . . . . . . . . 11
			Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13
	1. Introduction		1. Introduction
	Some recent Internet technology developments relate to improvements		Some recent Internet technology developments relate to improvements
	in communications latency. For instance, improvements in radio		in communications latency. For instance, improvements in radio
	communications or the recent work in IETF transport, security, and		communications or the recent work in IETF transport, security, and
	web protocols.		web protocols.
	There are also potential applications where latency is potentially in		There are also potential applications where latency is potentially in
	a more significant role than it has traditionally been in Internet		a more significant role than it has traditionally been in Internet
	communications.		communications.
	New applications or technologies does not necessarily imply that		New applications or technologies do not necessarily imply that
	latency should be the main driving concern, or that any further		latency should be the main driving concern, or that any further
	efforts beyond those already ongoing are needed. Indeed, modern		efforts beyond those already ongoing are needed. Indeed, modern
	networking systems offer many tools for building low-latency		networking systems offer many tools for building low-latency
	networks, across the stack. At the IETF, for instance, there has		networks, across the stack. At the IETF, for instance, there has
	been a recent increase in work related to transport, security, and		been a recent increase in work related to transport, security, and
	web application protocols, in part to make significant improvements		web application protocols, in part to make significant improvements
	in latency and connection set-up times. Similar efforts in other		in latency and connection set-up times. Similar efforts for other
	parts of the stack exist in 3GPP, IEEE, and other standards		components of communications technology exist in 3GPP, IEEE, and
	organisations.		other standards organisations.
	Despite a large number of specific developments, it may be		Despite a large number of specific developments, it may be
	interesting to view the developments from a system viewpoint, and to		interesting to view the developments from a system viewpoint, and to
	consider the potential future stresses that the strive for low-		consider the potential future stresses that the strive for low-
	latency support for applications may bring.		latency support for applications may bring.
	The rest of this memo is organised as follows. Section 2 discusses		The rest of this memo is organised as follows. Section 2 discusses
	potential applications for low-latency communications. Section 4		potential applications for low-latency communications. Section 4
	reviews some of the recent work across the stack, relating to latency		reviews some of the recent work across the stack, relating to latency
	improvements. Finally, Section 5 discusses some of the implications		improvements. Finally, Section 5 discusses some of the implications
	(and non-implications) from an architectural perspective.		(and non-implications) from an architectural perspective.
	2. Applications with Special Focus on Low Latency		2. Applications with Special Focus on Low Latency
	Most Internet applications enjoy significant benefits from low-		Most Internet applications enjoy significant benefits from low-
	latency communications.		latency communications in the form of faster response times and
			higher bandwidth communications enabled by transport protocol
			behaviour [RFC1323].
	There are also potential applications where latency is potentially in		There are also potential applications where latency is potentially in
	an even more significant role. For instance, embedding		an even more significant role. For instance, embedding
	communications technology in automation or traffic systems, or		communications technology in automation or traffic systems, or
	consumer applications such as augmented or virtual reality where		consumer applications such as augmented or virtual reality where
	advance buffering may not be feasible.		advance buffering may not be feasible.
	Many of the Internet-of-Things and critical services use cases in 5G,		Many of the Internet-of-Things and critical services use cases in 5G,
	for instance, have been listed as requiring low latency and high		for instance, have been listed as requiring low latency and high
	reliability for communications [ER2015] [HU2015] [NGMN2015] [NO2015]		reliability for communications [ER2015] [HU2015] [NGMN2015] [NO2015]
	skipping to change at page 3, line 38		skipping to change at page 3, line 45
	as electricity networks, connected automation systems in factories,		as electricity networks, connected automation systems in factories,
	remote control of machinery such as mining equipment, or embedded		remote control of machinery such as mining equipment, or embedded
	technology in road or railway traffic.		technology in road or railway traffic.
	The different applications vary in terms of their needs. Some may be		The different applications vary in terms of their needs. Some may be
	very focused on high-speed local area communication, others need to		very focused on high-speed local area communication, others need to
	connect at optimal speed over a wide-area network, and yet others		connect at optimal speed over a wide-area network, and yet others
	need to find the right ways to provide global services without		need to find the right ways to provide global services without
	incurring unreasonable delays.		incurring unreasonable delays.
			For these reasons it is difficult to specify what "low latency" means
			in terms of specific delays. Applications and network scenarios
			differ. Reaching a 50ms latency may be enough for some applications
			while others may require 50us. Obviously, latency is ultimately
			limited by physics, location, and topology. Individual link
			characteristics are important, but system level communication needs
			both in terms of what is being communicated and between what parties
			matter more.
	Note that when we say "low-latency capabilities", there is no intent		Note that when we say "low-latency capabilities", there is no intent
	to imply any specific implementation of those capabilities. In		to imply any specific implementation of those capabilities. In
	particular, we look at the low-latency requirements from a broader		particular, we look at the low-latency requirements from a broader
	perspective than Quality-of-Service guarantees or separating traffic		perspective than Quality-of-Service guarantees or separating traffic
	onto different classes. Indeed, while today's virtualisation and		onto different classes. Indeed, while today's virtualisation and
	software-driven technologies give us more tools to deal with those		software-driven technologies give us more tools to deal with those
	kinds of arrangements as well, past experience on deploying Quality-		kinds of arrangements as well, past experience on deploying Quality-
	of-Service mechanisms in the Internet should give us pause [CC2015].		of-Service mechanisms in the Internet should give us pause [CC2015].
	It is not the purpose of this memo to analyse the application		It is not the purpose of this memo to analyse the application
	requirements for low-latency applications much further; for our		requirements for low-latency applications much further; for our
	purposes it suffices to note that there are applications that are		purposes it suffices to note that there are applications that are
	enabled by low-latency capabilities of the underlying network		enabled by low-latency capabilities of the underlying network
	infrastructure.		infrastructure.
	3. Role of Low-Latency vs. Other Communications		3. Role of Low-Latency vs. Other Communications
	There are some limited applications that rely solely on local		There are some limited applications that rely solely on local
	communication. One example of such an application is vehicles		communication. One example of such an application is vehicles
	communicating braking status to nearby ones. However, many		communicating braking status to nearby ones.
	applications will include also wide-area communication. If the
	factory automation machines are not talking other than with		Also, while many applications run in the global Internet, some are
			designed for specialised networks that may not have have full or even
			any Internet connectivity, but yet use IP technology.
			However, many applications will include also wide-area communication.
			If the factory automation machines are not talking other than with
	themselves, at least their control systems are doing so in order to		themselves, at least their control systems are doing so in order to
	ensure parts orders, monitoring and maintenance by equipment		ensure parts orders, monitoring and maintenance by equipment
	manufacturers, and so on. This does not imply that these perhaps		manufacturers, and so on. This does not imply that these perhaps
	critical applications are openly accessible from the Internet, but		critical applications are openly accessible from the Internet, but
	many of them are likely to communicate outside their immediate		many of them are likely to communicate outside their immediate
	surroundings.		surroundings.
	Many applications also rely on wide-area connectivity for software		Many applications also rely on wide-area connectivity for software
	updates.		updates.
	As a result, this document recommends that when building		As a result, this document recommends that when building
	architectures for low-latency applications it is important to take		architectures for low-latency applications it is important to take
	into account that these applications can also benefit from		into account that these applications can also benefit from
	communications elsewhere.		communications elsewhere. Or at the very least, the existence of a
			specialised communications link or network should not be immediately
			taken to mean that no other communications are needed.
	4. Selected Improvements to Communications Latency		4. Selected Improvements to Communications Latency
	It should be noted that latency is a very broad topic in		It should be noted that latency is a very broad topic in
	communications protocol design, almost as broad as "security", or		communications protocol design, almost as broad as "security", or
	even "correctness".		even "correctness".
	Implementation techniques to satisfy these requirements vary, some		Implementation techniques to satisfy these requirements vary, some
	applications can be built with sufficient fast local networking		applications can be built with sufficient fast local networking
	capabilities, others may require, for instance, building world-wide,		capabilities, others may require, for instance, building world-wide,
	distributed content delivery mechanisms.		distributed content delivery mechanisms.
	Modern networking systems offer many tools for building low-latency		Modern networking systems offer many tools for building low-latency
	networks, across the stack. from highly optimised individual protocol		networks, across the stack. from highly optimised individual protocol
	components [I-D.ietf-tls-tls13] [I-D.ietf-quic-transport] [RFC7540]		components [I-D.ietf-tls-tls13] [I-D.ietf-quic-transport] [RFC7413]
	to software controlled, virtualised and tailored network functions		[RFC7540] to software controlled, virtualised and tailored network
	[NFV2012] [I-D.ietf-sfc-nsh] [OF2008]. Data- and software-driven		functions [NFV2012] [I-D.ietf-sfc-nsh] [OF2008]. Data- and software-
	network managment and orchestration tools enable networks to be built		driven network management and orchestration tools enable networks to
	to serve particular needs.		be built to serve particular needs.
	Across the stack there are also many other tools, as well as tools		Across the stack there are also many other tools, as well as tools
	being in development, e.g., a new transport design [L4S] at the IETF.		being in development, e.g., a new transport design [L4S] at the IETF.
	On the lower layers, improvements in radio communications are being		On the lower layers, improvements in radio communications are being
	made. For instance, the IEEE 802.1 Time-Sensitive Networking Task		made. For instance, the IEEE 802.1 Time-Sensitive Networking Task
	Group [TSN8021] has worked to define precise time synchronization		Group [TSN8021] has worked to define precise time synchronization
	mechanisms for a local area network, and scheduling mechanisms to		mechanisms for a local area network, and scheduling mechanisms to
	enable different classes of traffic to use the same network while		enable different classes of traffic to use the same network while
	minimising jitter and latency. At the IETF, the DETNET working group		minimising jitter and latency. At the IETF, the DETNET working group
	is taking these capabilities and applying them for layer 3 networking		is taking these capabilities and applying them for layer 3 networking
	[DETNET].		[DETNET].
	The 3GPP 5G requirements for next-generation access technology are		The 3GPP 5G requirements for next-generation access technology are
	stringent, and are leading to the optimization of the radio		stringent, and are leading to the optimization of the radio
	interfaces. The requirements specify a one-way latency limit of		interfaces. The requirements specify a one-way latency limit of
	0.5ms for ultra-reliable low-latency communications [TS38913].		0.5ms for ultra-reliable low-latency communications [TS38913]. But
			again, mere latency numbers mean very little without the context of a
			system and what an application needs to communicate and with whom.
	5. Architectural Considerations		5. Architectural Considerations
	Despite a large number of specific developments, it may be		Despite a large number of specific developments, it may be
	interesting to view the developments from a system viewpoint, and to		interesting to view the developments from a system viewpoint, and to
	consider the potential future stresses that the strive for low-		consider the potential future stresses that the strive for low-
	latency support for applications may bring.		latency support for applications may bring.
	5.1. Background		5.1. Background
	skipping to change at page 6, line 42		skipping to change at page 7, line 25
	number of different sensors and actuators.		number of different sensors and actuators.
	We cannot change the speed of light, and a single exchange with		We cannot change the speed of light, and a single exchange with
	another part of the world may result in a 100ms delay, or about 200		another part of the world may result in a 100ms delay, or about 200
	times longer than the expected 5G radio link delay for critical		times longer than the expected 5G radio link delay for critical
	applications. It is clear that designing applications from a system		applications. It is clear that designing applications from a system
	perspective is very important.		perspective is very important.
	5.2. Implications		5.2. Implications
			This section discusses a selected set of architectural effects and
			design choices within applications that desire low latency
			communications.
			5.2.1. Service Distribution
	As noted above, low-latency applications need to pay particular		As noted above, low-latency applications need to pay particular
	attention to the placement of services in the global network.		attention to the placement of services in the global network.
	Operations that are on the critical path for the low-latency aspects		Operations that are on the critical path for the low-latency aspects
	of an application are unlikely to work well if those communications		of an application are unlikely to work well if those communications
	need to traverse half of the Internet.		need to traverse half of the Internet.
	Many widely used services are already distributed and replicated		Many widely used services are already distributed and replicated
	throughout the world, to minimise communications latency. But many		throughout the world, to minimise communications latency. But many
	other services are not distributed in this manner. For low-latency		other services are not distributed in this manner. For low-latency
	applications such distribution becomes necessary. Hosting a global		applications such distribution becomes necessary. Hosting a global
	service in one location is not feasible due to latency, even when		service in one location is not feasible due to latency, even when
	from a scale perspective a single server might otherwise suffice for		from a scale perspective a single server might otherwise suffice for
	the service.		the service.
	Content-Delivery Networks (CDNs) and similar arrangements are likely		Content-Delivery Networks (CDNs) and similar arrangements are likely
	to flourish because of this. In the most extreme cases, edge		to flourish because of this. These arrangements can bring content
	computing services are needed.		close to end-users, and have a significant impact on latency.
			Typical CDN arrangements provide services that are on a global scale
			nearby, e.g., in the same country or even at the ISP's datacenter.
			Today's CDNs are of course just one form of distributed service
			implementation. Previous generations, such as web caching, have
			existed as well, and it is likely that the current arrangements will
			evolve in the future. CDN evolution is also naturally affected not
			only by the need to provide services closer to the user, but also
			through the fine-grained control and visibility mechanisms that it
			gives to the content owners. Such factors continue to affect also
			future evolution, e.g., any information-centric networking solutions
			that might emerge.
			5.2.2. Edge Computing
			Recent efforts in "edge computing" takes the CDN type service
			placement even further by providing services near the users. This
			would enable more extreme uses cases where latency from, say, ISP
			datacenter to the users is considered too high. An important
			consideration is what is considered an edge, however. From an
			Internet perspective edge usually refers to the IP point of presence
			or the first IP hop. But given the centralised nature of many access
			networks, some of the discussions around the use of edge computing
			also involve components at the edge that are much closer to user than
			the first IP hop. Special arrangements are needed to enable direct
			IP connectivity from the user equipment to these components.
			5.2.3. Routing and tunnels
	How the communications are routed also matters. For instance,		How the communications are routed also matters. For instance,
	architectures based on tunneling to a central point may incur extra		architectures based on tunneling to a central point may incur extra
	delay. One way to address this pressure is to use SDN- and		delay. One way to address this pressure is to use SDN- and
	virtualisation-based networks that can be provisioned in the desired		virtualisation-based networks that can be provisioned in the desired
	manner, so that, for instance, instances of tunnel servers can be		manner, so that, for instance, instances of tunnel servers can be
	placed in the topologically optimal place for a particular		placed in the topologically optimal place for a particular
	application.		application.
			5.2.4. Alternative Paths and Control Tension
	Recent developments in multipath transport protocols [RFC6824] also		Recent developments in multipath transport protocols [RFC6824] also
	provide application- and service-level control of some of the		provide application- and service-level control of some of the
	networking behaviour. There is tension between application control		networking behaviour. Similar choices among alternative paths also
	(often by content providers) and network control (often by network		exist in simpler techniques, ranging from server selection algorithms
	operators).		to IPv6 "Happy Eyeballs" algorithms [RFC6555]. In all of these cases
			an application makes some observations of the environment and decides
			to use the an alternative path or target that is perceived to be best
			suited for the applications's needs.
			In all of these multipath and alternative selection techniques there
			is tension between application control (often by content providers)
			and network control (often by network operators).
	One special case where that tension has appeared in the past is		One special case where that tension has appeared in the past is
	whether there should be ways to provide information from applications		whether there should be ways to provide information from applications
	to networks on how packets should be treated. This was extensively		to networks on how packets should be treated. This was extensively
	discussed during the discussion stemming from implications of		discussed during the discussion stemming from implications of
	increased use of encryption in the Internet, and how that affects		increased use of encryption in the Internet, and how that affects
	operators [I-D.nrooney-marnew-report].		operators [I-D.nrooney-marnew-report].
	Another case where there is tension is between mechanisms designed		Another case where there is tension is between mechanisms designed
	for a single link or network vs. end-to-end mechanisms. Many of the		for a single link or network vs. end-to-end mechanisms. Many of the
	stated requirements for low-latency applications are explicitly about		stated requirements for low-latency applications are explicitly about
	end-to-end characteristics and capabilities. Yet, the two mechanisms		end-to-end characteristics and capabilities. Yet, the two mechanisms
	are very different, and most of the deployment difficulties reported		are very different, and most of the deployment difficulties reported
	in [CC2015] relate to end-to-end mechanisms.		in [CC2015] relate to end-to-end mechanisms.
	Finally, in the search for even faster connection setup times one		Note that some of the multipath techniques can be used either by
	obvious technique is cross-layer optimisation. We have seen some of		endpoints or by the network. Proxy-based Multipath TCP is one
	this in the IETF in the rethinking of the layers for transport,		example of this [I-D.boucadair-mptcp-plain-mode].
	transport layer security, and application framework protocols.
			5.2.5. Cross-Layer Optimisations
			In the search for even faster connection setup times one obvious
			technique is cross-layer optimisation. We have seen some of this in
			the IETF in the rethinking of the layers for transport, transport
			layer security, and application framework protocols. By taking into
			account the protocol layer interactions or even bundling the protocol
			design together, it is relatively easy to optimise the connection
			setup time, as evidenced by recent efforts to look for "0-RTT"
			designs in various protocols.
	But while cross-layer optimisation can bring benefits, it also has		But while cross-layer optimisation can bring benefits, it also has
	downsides. In particular, it connects different parts of the stack		downsides. In particular, it connects different parts of the stack
	in additional ways. This can lead to difficulties in further		in additional ways. This can lead to difficulties in further
	evolution of the technology, if done wrong.		evolution of the technology, if done wrong.
	In the case of the IETF transport protocol evolution, significant		In the case of the IETF transport protocol evolution, significant
	improvements were made to ensure better evolvability of the		improvements were made to ensure better evolvability of the
	protocols than what we've experienced with TCP, starting from an		protocols than what we've experienced with TCP, starting from an
	ability to implement the new protocols in applications rather than		ability to implement the new protocols in applications rather than
	in the kernel.		in the kernel.
			While the connection setup is an obvious example, cross-layer
			optimisations are not limited to them. Interfaces between
			application, transport, networking, and link layers can provide
			information and set parameters that improve latency. For instance,
			setting DSCP values or requesting a specialised L2 service for a
			particular application.
	The effects of badly designed cross-layer optimisation are a		The effects of badly designed cross-layer optimisation are a
	particular form of Internet ossification. The general networking		particular form of Internet ossification. The general networking
	trend, however, is for greater flexibility and programmability.		trend, however, is for greater flexibility and programmability.
	Arguably, the ease at which networks can evolve is probably even more		Arguably, the ease at which networks can evolve is probably even more
	important than their specific characteristics.		important than their specific characteristics.
			These comments about cross-layer optimisation should not be
			interpreted to mean that protocol design should not take into account
			how other layers behave. The IETF has a long tradition of discussing
			link layer design implications for Internet communications (see,
			e.g., the results of the PILC working group [RFC3819].
	5.3. Recommendations for Further Work		5.3. Recommendations for Further Work
	Low-latency applications continue to be a hot topic in networking.		Low-latency applications continue to be a hot topic in networking.
	The following topics in particular deserve further work from an		The following topics in particular deserve further work from an
	architectural point of view:		architectural point of view:
	o Application architectures for globally connected but low-latency		o Application architectures for globally connected but low-latency
	services.		services.
	o What are the issues with inter-domain Quality-of-Service		o What are the issues with inter-domain Quality-of-Service
	skipping to change at page 8, line 43		skipping to change at page 10, line 46
	o Inter-organisatorial matters, e.g., to what extent different		o Inter-organisatorial matters, e.g., to what extent different
	standards organisations need to talk about low latency effects and		standards organisations need to talk about low latency effects and
	ongoing work, to promote system-level understanding?		ongoing work, to promote system-level understanding?
	Overall, this memo stresses the importance of the system-level		Overall, this memo stresses the importance of the system-level
	understanding of Internet applications and their latency issues.		understanding of Internet applications and their latency issues.
	Efforts to address specific sub-issues are unlikely to be fruitful		Efforts to address specific sub-issues are unlikely to be fruitful
	without a holistic plan.		without a holistic plan.
			In the authors' opinion, the most extreme use cases (e.g., 1ms or
			smaller latencies) are not worth building general-purpose networks
			for. But having the necessary tools so that networks can be flexible
			for the more general cases is very useful, as there are many
			applications that can benefit from the added flexibility. The key
			tools for this include ability to manage network function placement
			and topology.
	6. Acknowledgements		6. Acknowledgements
	The author would like to thank Brian Trammell, Mirja Kuehlewind,		The author would like to thank Brian Trammell, Mirja Kuehlewind,
	Linda Dunbar, Goran Rune, Ari Keranen, Jeff Tantsura, Stephen		Linda Dunbar, Goran Rune, Ari Keranen, Jeff Tantsura, James Kempf,
	Farrell, and many others for interesting discussions in this problem		Stephen Farrell, Mohamed Boucadair, Kumar Balachandran, Goran AP
			Eriksson, and many others for interesting discussions in this problem
	space.		space.
	The author would also like to acknowledge the important contribution		The author would also like to acknowledge the important contribution
	that [I-D.dunbar-e2e-latency-arch-view-and-gaps] made in this topic.		that [I-D.dunbar-e2e-latency-arch-view-and-gaps] made in this topic.
	7. Informative References		7. Informative References
	[CC2015] claffy, kc. and D. Clark, "Adding Enhanced Services to the		[CC2015] claffy, kc. and D. Clark, "Adding Enhanced Services to the
	Internet: Lessons from History", September 2015		Internet: Lessons from History", September 2015
	(https://www.caida.org/publications/papers/2015/		(https://www.caida.org/publications/papers/2015/
	skipping to change at page 9, line 25		skipping to change at page 11, line 36
	[ER2015] Yilmaz, O., "5G Radio Access for Ultra-Reliable and Low-		[ER2015] Yilmaz, O., "5G Radio Access for Ultra-Reliable and Low-
	(https://www.ericsson.com/research-blog/5g/5g-radio-		(https://www.ericsson.com/research-blog/5g/5g-radio-
	access-for-ultra-reliable-and-low-latency-		access-for-ultra-reliable-and-low-latency-
	communications/).		communications/).
	[HU2015] "5G Vision: 100 Billion connections, 1 ms Latency, and 10		[HU2015] "5G Vision: 100 Billion connections, 1 ms Latency, and 10
	Gbps Throughput", Huawei 2015		Gbps Throughput", Huawei 2015
	(http://www.huawei.com/minisite/5g/en/defining-5g.html).		(http://www.huawei.com/minisite/5g/en/defining-5g.html).
			[I-D.boucadair-mptcp-plain-mode]
			Boucadair, M., Jacquenet, C., Bonaventure, O., Behaghel,
			D., stefano.secci@lip6.fr, s., Henderickx, W., Skog, R.,
			Vinapamula, S., Seo, S., Cloetens, W., Meyer, U.,
			Contreras, L., and B. Peirens, "Extensions for Network-
			Assisted MPTCP Deployment Models", draft-boucadair-mptcp-
			plain-mode-10 (work in progress), March 2017.
	[I-D.dunbar-e2e-latency-arch-view-and-gaps]		[I-D.dunbar-e2e-latency-arch-view-and-gaps]
	Dunbar, L., "Architectural View of E2E Latency and Gaps",		Dunbar, L., "Architectural View of E2E Latency and Gaps",
	draft-dunbar-e2e-latency-arch-view-and-gaps-01 (work in		draft-dunbar-e2e-latency-arch-view-and-gaps-01 (work in
	progress), March 2017.		progress), March 2017.
	[I-D.ietf-quic-transport]		[I-D.ietf-quic-transport]
	Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed		Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
	and Secure Transport", draft-ietf-quic-transport-02 (work		and Secure Transport", draft-ietf-quic-transport-04 (work
	in progress), March 2017.		in progress), June 2017.
	[I-D.ietf-sfc-nsh]		[I-D.ietf-sfc-nsh]
	Quinn, P. and U. Elzur, "Network Service Header", draft-		Quinn, P. and U. Elzur, "Network Service Header", draft-
	ietf-sfc-nsh-12 (work in progress), February 2017.		ietf-sfc-nsh-13 (work in progress), June 2017.
	[I-D.ietf-tls-tls13]		[I-D.ietf-tls-tls13]
	Rescorla, E., "The Transport Layer Security (TLS) Protocol		Rescorla, E., "The Transport Layer Security (TLS) Protocol
	Version 1.3", draft-ietf-tls-tls13-20 (work in progress),		Version 1.3", draft-ietf-tls-tls13-20 (work in progress),
	April 2017.		April 2017.
	[I-D.nrooney-marnew-report]		[I-D.nrooney-marnew-report]
	Rooney, N., "IAB Workshop on Managing Radio Networks in an		Rooney, N., "IAB Workshop on Managing Radio Networks in an
	Encrypted World (MaRNEW) Report", draft-nrooney-marnew-		Encrypted World (MaRNEW) Report", draft-nrooney-marnew-
	report-02 (work in progress), August 2016.		report-03 (work in progress), June 2017.
	[IMT2020] "Framework and overall objectives of the future		[IMT2020] "Framework and overall objectives of the future
	development of IMT for 2020 and beyond", ITU		development of IMT for 2020 and beyond", ITU
	Recommendation M.2083-0, September 2015		Recommendation M.2083-0, September 2015
	(http://www.itu.int/rec/R-REC-M.2083-0-201509-I/en).		(http://www.itu.int/rec/R-REC-M.2083-0-201509-I/en).
	[L4S] "Low Latency Low Loss Scalable throughput (L4S) Birds-of-		[L4S] "Low Latency Low Loss Scalable throughput (L4S) Birds-of-
	(https://datatracker.ietf.org/wg/l4s/charter/).		(https://datatracker.ietf.org/wg/l4s/charter/).
	[NFV2012] "Network Functions Virtualisation - Introductory White		[NFV2012] "Network Functions Virtualisation - Introductory White
	skipping to change at page 10, line 38		skipping to change at page 13, line 9
	[OF2008] McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G.,		[OF2008] McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G.,
	Peterson, L., Rexford, J., Shenker, S., and J. Turner,		Peterson, L., Rexford, J., Shenker, S., and J. Turner,
	"OpenFlow: Enabling Innovation in Campus Networks", ACM		"OpenFlow: Enabling Innovation in Campus Networks", ACM
	SIGCOMM Computer Communication Review, Volume 38, Issue 2,		SIGCOMM Computer Communication Review, Volume 38, Issue 2,
	pp. 69-74 2008.		pp. 69-74 2008.
	[QU2016] "Leading the world to 5G", Qualcomm, February 2016		[QU2016] "Leading the world to 5G", Qualcomm, February 2016
	(https://www.qualcomm.com/media/documents/files/qualcomm-		(https://www.qualcomm.com/media/documents/files/qualcomm-
	5g-vision-presentation.pdf).		5g-vision-presentation.pdf).
			[RFC1323] Jacobson, V., Braden, R., and D. Borman, "TCP Extensions
			for High Performance", RFC 1323, DOI 10.17487/RFC1323, May
			1992, <http://www.rfc-editor.org/info/rfc1323>.
			[RFC3819] Karn, P., Ed., Bormann, C., Fairhurst, G., Grossman, D.,
			Ludwig, R., Mahdavi, J., Montenegro, G., Touch, J., and L.
			Wood, "Advice for Internet Subnetwork Designers", BCP 89,
			RFC 3819, DOI 10.17487/RFC3819, July 2004,
			<http://www.rfc-editor.org/info/rfc3819>.
			[RFC6555] Wing, D. and A. Yourtchenko, "Happy Eyeballs: Success with
			Dual-Stack Hosts", RFC 6555, DOI 10.17487/RFC6555, April
			2012, <http://www.rfc-editor.org/info/rfc6555>.
	[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,		[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
	"TCP Extensions for Multipath Operation with Multiple		"TCP Extensions for Multipath Operation with Multiple
	Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,		Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,
	<http://www.rfc-editor.org/info/rfc6824>.		<http://www.rfc-editor.org/info/rfc6824>.
			[RFC7413] Cheng, Y., Chu, J., Radhakrishnan, S., and A. Jain, "TCP
			Fast Open", RFC 7413, DOI 10.17487/RFC7413, December 2014,
			<http://www.rfc-editor.org/info/rfc7413>.
	[RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext		[RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext
	Transfer Protocol Version 2 (HTTP/2)", RFC 7540,		Transfer Protocol Version 2 (HTTP/2)", RFC 7540,
	DOI 10.17487/RFC7540, May 2015,		DOI 10.17487/RFC7540, May 2015,
	<http://www.rfc-editor.org/info/rfc7540>.		<http://www.rfc-editor.org/info/rfc7540>.
	[TS38913] "3rd Generation Partnership Project; Technical		[TS38913] "3rd Generation Partnership Project; Technical
	Specification Group Radio Access Network; Study on		Specification Group Radio Access Network; Study on
	Scenarios and Requirements for Next Generation Access		Scenarios and Requirements for Next Generation Access
	Technologies; (Release 14)", 3GPP Technical Report TR		Technologies; (Release 14)", 3GPP Technical Report TR
	(https://portal.3gpp.org/desktopmodules/Specifications/		(https://portal.3gpp.org/desktopmodules/Specifications/
	SpecificationDetails.aspx?specificationId=2996).		SpecificationDetails.aspx?specificationId=2996).
	[TSN8021] "Time-Sensitive Networking Task Group", IEEE		[TSN8021] "Time-Sensitive Networking Task Group", IEEE
	(http://www.ieee802.org/1/pages/tsn.html).		(http://www.ieee802.org/1/pages/tsn.html).
	Author's Address		Authors' Addresses
	Jari Arkko		Jari Arkko
	Ericsson		Ericsson
	Kauniainen 02700		Kauniainen 02700
	Finland		Finland
	Email: jari.arkko@piuha.net		Email: jari.arkko@piuha.net
			Jeff Tantsura
			Individual
			Email: jefftant.ietf@gmail.com
End of changes. 30 change blocks.
	43 lines changed or deleted		169 lines changed or added
This html diff was produced by rfcdiff 1.34. The latest version is available from http://tools.ietf.org/tools/rfcdiff/