Spatial Composition of MetricsAT&T Labs200 Laurel Avenue SouthMiddletown,NJ07748USA+1 732 420 1571+1 732 368 1192acmorton@att.comhttp://home.comcast.net/~acmacm/France Telecom Division R&D2 avenue Pierre MarzinLannionF-22307France+33 2 96 05 18 52emile.stephan@orange-ftgroup.comThis memo utilizes IPPM metrics that are applicable to both complete
paths and sub-paths, and defines relationships to compose a complete
path metric from the sub-path metrics with some accuracy w.r.t. the
actual metrics. This is called Spatial Composition in RFC 2330. The memo
refers to the Framework for Metric Composition, and provides background
and motivation for combining metrics to derive others. The descriptions
of several composed metrics and statistics follow.The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.In this memo, the characters "<=" should be read as "less than or
equal to" and ">=" as "greater than or equal to".Thus far, the following people have contributed useful ideas,
suggestions, or the text of sections that have been incorporated into
this memo:- Phil Chimento <vze275m9@verizon.net>- Reza Fardid <RFardid@Covad.COM>- Roman Krzanowski <roman.krzanowski@verizon.com>- Maurizio Molina <maurizio.molina@dante.org.uk>- Lei Liang <L.Liang@surrey.ac.uk>- Dave Hoeflin <dhoeflin@att.com>The IPPM framework describes two forms
of metric composition, spatial and temporal. The new composition
framework expands
and further qualifies these original forms into three categories. This
memo describes Spatial Composition, one of the categories of metrics
under the umbrella of the composition framework.Spatial composition encompasses the definition of performance metrics
that are applicable to a complete path, based on metrics collected on
various sub-paths.The main purpose of this memo is to define the deterministic
functions that yield the complete path metrics using metrics of the
sub-paths. The effectiveness of such metrics is dependent on their
usefulness in analysis and applicability with practical measurement
methods.The relationships may involve conjecture, and lists four points that the metric definitions
should include: the specific conjecture applied to the metric and assumptions of
the statistical model of the process being measured (if any, see
section 12),a justification of the practical utility of the composition in
terms of making accurate measurements of the metric on the path,a justification of the usefulness of the composition in terms of
making analysis of the path using A-frame concepts more effective,
andan analysis of how the conjecture could be incorrect.Also, gives an example using the
conjecture that the delay of a path is very nearly the sum of the delays
of the exchanges and clouds of the corresponding path digest. This
example is particularly relevant to those who wish to assess the
performance of an Inter-domain path without direct measurement, and the
performance estimate of the complete path is related to the measured
results for various sub-paths instead.Approximate functions between the sub-path and complete path metrics
are useful, with knowledge of the circumstances where the relationships
are/are not applicable. For example, we would not expect that delay
singletons from each sub-path would sum to produce an accurate estimate
of a delay singleton for the complete path (unless all the delays were
essentially constant - very unlikely). However, other delay statistics
(based on a reasonable sample size) may have a sufficiently large set of
circumstances where they are applicable.One-way metrics defined in other IPPM RFCs all assume that the
measurement can be practically carried out between the source and the
destination of the interest. Sometimes there are reasons that the
measurement can not be executed from the source to the destination.
For instance, the measurement path may cross several independent
domains that have conflicting policies, measurement tools and methods,
and measurement time assignment. The solution then may be the
composition of several sub-path measurements. This means each domain
performs the One-way measurement on a sub path between two nodes that
are involved in the complete path following its own policy, using its
own measurement tools and methods, and using its own measurement
timing. Under the appropriate conditions, one can combine the sub-path
One-way metric results to estimate the complete path One-way
measurement metric with some degree of accuracy.For the primary IPPM metrics of Loss, Delay, and Delay Variation,
this memo gives a set of metrics that can be composed from the same or
similar sub-path metrics. This means that the composition function may
utilize: the same metric for each sub-path;multiple metrics for each sub-path (possibly one that is the
same as the complete path metric);a single sub-path metric that is different from the complete
path metric;different measurement techniques like active and passive
(recognizing that PSAMP WG will define capabilities to sample
packets to support measurement).We note a possibility: Using a complete path metric and all
but one sub-path metric to infer the performance of the missing
sub-path, especially when the "last" sub-path metric is missing.
However, such de-composition calculations, and the corresponding set
of issues they raise, are beyond the scope of this memo.The new composition framework requires the
specification of the applicable circumstances for each metric. In
particular, each section addresses whether the metric:Requires the same test packets to traverse all sub-paths, or may
use similar packets sent and collected separately in each
sub-path.Requires homogeneity of measurement methodologies, or can allow a
degree of flexibility (e.g., active or passive methods produce the
"same" metric). Also, the applicable sending streams will be
specified, such as Poisson, Periodic, or both.Needs information or access that will only be available within an
operator's domain, or is applicable to Inter-domain composition.Requires synchronized measurement time intervals in all sub-paths,
or largely overlapping, or no timing requirements.Requires assumption of sub-path independence w.r.t. the metric
being defined/composed, or other assumptions.Has known sources of inaccuracy/error, and identifies the
sources.In practice, when measurements cannot be initiated on a sub-path
(and perhaps the measurement system gives up during the test
interval), then there will not be a value for the sub-path reported,
and the entire test result SHOULD be recorded as "undefined". This
case should be distinguished from the case where the measurement
system continued to send packets throughout the test interval, but all
were declared lost.When a composed metric requires measurements from sub paths A, B,
and C, and one or more of the sub-path results are undefined, then the
composed metric SHOULD also be recorded as undefined.To reduce the redundant information presented in the detailed metrics
sections that follow, this section presents the specifications that are
common to two or more metrics. The section is organized using the same
subsections as the individual metrics, to simplify comparisons.Also, the following index variables represent the following:m = index for packets sentn = index for packets receiveds = index for involved sub-pathsAll metrics use the Type-P convention as described in . The rest of the name is unique to each
metric.Src, the IP address of a hostDst, the IP address of a hostT, a time (start of test interval)Tf, a time (end of test interval)lambda, a rate in reciprocal seconds (for Poisson
Streams)incT, the nominal duration of inter-packet interval, first
bit to first bit (for Periodic Streams)T0, a time that MUST be selected at random from the interval
[T, T+dT] to start generating packets and taking measurements
(for Periodic Streams)TstampSrc, the wire time of the packet as measured at
MP(Src)TstampDst, the wire time of the packet as measured at
MP(Dst), assigned to packets that arrive within a "reasonable"
time.Tmax, a maximum waiting time for packets at the destination,
set sufficiently long to disambiguate packets with long delays
from packets that are discarded (lost), thus the distribution of
delay is not truncated.M, the total number of packets sent between T0 and TfN, the total number of packets received at Dst (sent between
T0 and Tf)S, the number of sub-paths involved in the complete Src-Dst
pathThis section is unique for every metric.This section is unique for every metric.This section is unique for every metric.This section is unique for every metric.This section is unique for each metric.It is sometimes impractical to conduct active measurements
between every Src-Dst pair. Since the full mesh of N measurement
points grows as N x N, the scope of measurement may be limited by
testing resources.There may be varying limitations on active testing in different
parts of the network. For example, it may not be possible to collect
the desired sample size in each test interval when access link speed
is limited, because of the potential for measurement traffic to
degrade the user traffic performance. The conditions on a low-speed
access link may be understood well-enough to permit use of a small
sample size/rate, while a larger sample size/rate may be used on
other sub-paths.Also, since measurement operations have a real monetary cost,
there is value in re-using measurements where they are applicable,
rather than launching new measurements for every possible
source-destination pair.The measurement packets, each having source and destination
addresses intended for collection at edges of the sub-path, may
take a different specific path through the network equipment and
links when compared to packets with the source and destination
addresses of the complete path. Examples sources of parallel paths
include Equal Cost Multi-Path and parallel (or bundled) links.
Therefore, the performance estimated from the composition of
sub-path measurements may differ from the performance experienced
by packets on the complete path. Multiple measurements employing
sufficient sub-path address pairs might produce bounds on the
extent of this error.We also note the possibility of re-routing during a measurement
interval, as it may affect the correspondence between packets
traversing the complete path and the sub-paths that were
"involved" prior to the re-route.Related to the case of an alternate path described above is the
case where elements in the measured path are unique to measurement
system connectivity. For example, a measurement system may use a
dedicated link to a LAN switch, and packets on the complete path
do not traverse that link. The performance of such a dedicated
link would be measured continuously, and its contribution to the
sub-path metrics SHOULD be minimized as a source of error.Measurements of sub-path performance may not cover all the
network elements on the complete path. For example, the network
exchange points might be excluded unless a cooperative measurement
is conducted. In this example, test packets on the previous
sub-path are received just before the exchange point and test
packets on the next sub-path are injected just after the same
exchange point. Clearly, the set of sub-path measurements SHOULD
cover all critical network elements in the complete path.At a specific point in time, no viable route exists between the
complete path source and destination. The routes selected for one
or more sub-paths therefore differs from the complete path.
Consequently, spatial composition may produce finite estimation of
a ground truth metric between a source and a destination, even
when the route between them is undefined.This section is unique for most metrics (see the metric-specific
sections).For delay-related metrics, One-way delay always depends on packet
size and link capacity, since it is measured in from first bit to last bit. If the size of
an IP packet changes (due to encapsulation for security reasons),
this will influence delay performance.Fragmentation is a major issue for compostion accuracy, since all
metrics require all fragments to arrive before proceeding, and
fragmented complete path performance is likely to be different from
performance with non-fragmented packets and composed metrics based
on non-fragmented sub-path measurements.The methodology:SHOULD use similar packets sent and collected separately in each
sub-path.Allows a degree of flexibility regarding test stream generation
(e.g., active or passive methods can produce an equivalent result,
but the lack of control over the source, timing and correlation of
passive measurements is much more challenging).Poisson and/or Periodic streams are RECOMMENDED.Applies to both Inter-domain and Intra-domain composition.SHOULD have synchronized measurement time intervals in all
sub-paths, but largely overlapping intervals MAY suffice.REQUIRES assumption of sub-path independence w.r.t. the metric
being defined/composed.This metric is a necessary element of Delay Composition metrics,
and its definition does not formally exist elsewhere in IPPM
literature.See the common parameters section above.Using the parameters above, we obtain the value of
Type-P-One-way-Delay singleton as per .For each packet [i] that has a finite One-way Delay (in other
words, excluding packets which have undefined one-way delay):Type-P-Finite-One-way-Delay-Poisson/Periodic-Stream[i] =FiniteDelay[i] = TstampDst - TstampSrcThe units of measure for this metric are time in seconds,
expressed in sufficiently low resolution to convey meaningful
quantitative information. For example, resolution of microseconds is
usually sufficient.The “Type-P-Finite-One-way-Delay” metric permits
calculation of the sample mean statistic. This resolves the problem
of including lost packets in the sample (whose delay is undefined),
and the issue with the informal assignment of infinite delay to lost
packets (practical systems can only assign some very large
value).The Finite-One-way-Delay approach handles the problem of lost
packets by reducing the event space. We consider conditional
statistics, and estimate the mean one-way delay conditioned on the
event that all packets in the sample arrive at the destination
(within the specified waiting time, Tmax). This offers a way to make
some valid statements about one-way delay, and at the same time
avoiding events with undefined outcomes. This approach is derived
from the treatment of lost packets in , and is similar to .All statistics defined in are
applicable to the finite one-way delay,and additional metrics are
possible, such as the mean (see below).This section describes a statistic based on the
Type-P-Finite-One-way-Delay-Poisson/Periodic-Stream metric.See the common parameters section above.We definewhere all packets n= 1 through N have finite singleton
delays.The units of measure for this metric are time in seconds,
expressed in sufficiently fine resolution to convey meaningful
quantitative information. For example, resolution of microseconds is
usually sufficient.The Type-P-Finite-One-way-Delay-Mean metric requires the
conditional delay distribution described in section 5.1.This metric, a mean, does not require additional statistics.The Type-P-Finite—Composite-One-way-Delay-Mean, or
CompMeanDelay, for the complete Source to Destination path can be
calculated from sum of the Mean Delays of all its S constituent
sub-paths.Then theThe mean of a sufficiently large stream of packets measured on
each sub-path during the interval [T, Tf] will be representative of
the ground truth mean of the delay distribution (and the
distributions themselves are sufficiently independent), such that
the means may be added to produce an estimate of the complete path
mean delay.It is assumed that the one-way delay distributions of the
sub-paths and the complete path are continuous. The mean of bi-modal
distributions have the unfortunate property that such a value may
never occur.See the common section.See the common section.If any of the sub-path distributions are bimodal, then the
measured means may not be stable, and in this case the mean will not
be a particularly useful statistic when describing the delay
distribution of the complete path.The mean may not be sufficiently robust statistic to produce a
reliable estimate, or to be useful even if it can be measured.If a link contributing non-negligible delay is erroneously
included or excluded, the composition will be in error.The requirements of the common section apply here as well.This section describes is a statistic based on the
Type-P-Finite-One-way-Delay-Poisson/Periodic-Stream metric, and the
composed metric based on that statistic.See the common parameters section above.We definewhere all packets n = 1 through N have finite singleton
delays.The units of measure for this metric are time in seconds,
expressed in sufficiently fine resolution to convey meaningful
quantitative information. For example, resolution of microseconds is
usually sufficient.The Type-P-Finite-One-way-Delay-Minimum metric requires the
conditional delay distribution described in section 5.1.3.This metric, a minimum, does not require additional
statistics.The Type-P-Finite—Composite-One-way-Delay-Minimum, or
CompMinDelay, for the complete Source to Destination path can be
calculated from sum of the Minimum Delays of all its S constituent
sub-paths.Then theThe minimum of a sufficiently large stream of packets measured on
each sub-path during the interval [T, Tf] will be representative of
the ground truth minimum of the delay distribution (and the
distributions themselves are sufficiently independent), such that
the minima may be added to produce an estimate of the complete path
minimum delay.It is assumed that the one-way delay distributions of the
sub-paths and the complete path are continuous.See the common section.See the common section.If the routing on any of the sub-paths is not stable, then the
measured minimum may not be stable. In this case the composite
minimum would tend to produce an estimate for the complete path that
may be too low for the current path.The requirements of the common section apply here as well.Same as section 4.1.1.Using the parameters above, we obtain the value of
Type-P-One-way-Packet-Loss singleton and stream as per .We obtain a sequence of pairs with elements as follows: TstampSrc, as aboveL, either zero or one, where L=1 indicates loss and L=0
indicates arrival at the destination within TstampSrc +
Tmax.Given the stream parameter M, the number of packets sent, we can
define the Empirical Probability of Loss Statistic (Ep), consistent
with Average Loss in [RFC2680], as follows:where all packets m = 1 through M have a value for L.The Type-P-One-way-Composite-Packet-Loss-Empirical-Probability,
or CompEp for the complete Source to Destination path can be
calculated by combining Ep of all its constituent sub-paths (Ep1,
Ep2, Ep3, ... Epn) asIf any Eps is undefined in a particular measurement interval,
possibly because a measurement system failed to report a value, then
any CompEp that uses sub-path s for that measurement interval is
undefined.The empirical probability of loss calculated on a sufficiently
large stream of packets measured on each sub-path during the
interval [T, Tf] will be representative of the ground truth
empirical loss probability (and the probabilities themselves are
sufficiently independent), such that the sub-path probabilities may
be combined to produce an estimate of the complete path empirical
loss probability.See the common section.See the common section.A concern for loss measurements combined in this way is that root
causes may be correlated to some degree.For example, if the links of different networks follow the same
physical route, then a single catastrophic event like a fire in a
tunnel could cause an outage or congestion on remaining paths in
multiple networks. Here it is important to ensure that measurements
before the event and after the event are not combined to estimate
the composite performance.Or, when traffic volumes rise due to the rapid spread of an
email-born worm, loss due to queue overflow in one network may help
another network to carry its traffic without loss.others...See the common section.This packet delay variation (PDV) metric is a necessary element of
Composed Delay Variation metrics, and its definition does not formally
exist elsewhere in IPPM literature.In addition to the parameters of section 4.1.1:TstampSrc[i], the wire time of packet[i] as measured at
MP(Src) (measurement point at the source)TstampDst[i], the wire time of packet[i] as measured at
MP(Dst), assigned to packets that arrive within a "reasonable"
time.B, a packet length in bitsF, a selection function unambiguously defining the packets
from the stream that are selected for the packet-pair
computation of this metric. F(first packet), the first packet of
the pair, MUST have a valid Type-P-Finite-One-way-Delay less
than Tmax (in other words, excluding packets which have
undefined one-way delay) and MUST have been transmitted during
the interval T, Tf. The second packet in the pair, F(second
packet) MUST be the packet with the minimum valid value of
Type-P-Finite-One-way-Delay for the stream, in addition to the
criteria for F(first packet). If multiple packets have equal
minimum Type-P-Finite-One-way-Delay values, then the value for
the earliest arriving packet SHOULD be used.MinDelay, the Type-P-Finite-One-way-Delay value for F(second
packet) given above.N, the number of packets received at the Destination meeting
the F(first packet) criteria.Using the definition above in section 5.1.2, we obtain the value
of Type-P-Finite-One-way-Delay-Poisson/Periodic-Stream[n], the
singleton for each packet[i] in the stream (a.k.a.
FiniteDelay[i]).For each packet[n] that meets the F(first packet) criteria given
above: Type-P-One-way-pdv-refmin-Poisson/Periodic-Stream[n] =PDV[n] = FiniteDelay[n] – MinDelaywhere PDV[i] is in units of time in seconds, expressed in
sufficiently fine resolution to convey meaningful quantitative
information. For example, resolution of microseconds is usually
sufficient.This metric produces a sample of delay variation normalized to
the minimum delay of the sample. The resulting delay variation
distribution is independent of the sending sequence (although
specific FiniteDelay values within the distribution may be
correlated, depending on various stream parameters such as packet
spacing). This metric is equivalent to the IP Packet Delay Variation
parameter defined in .We define the mean PDV as follows (where all packets n = 1
through N have a value for PDV[n]):We define the variance of PDV as follows:We define the skewness of PDV as follows:We define the Quantile of the IPDVRefMin sample as the value
where the specified fraction of singletons is less than the given
value.This section gives two alternative composition functions. The
objective is to estimate a quantile of the complete path delay
variation distribution. The composed quantile will be estimated
using information from the sub-path delay variation
distributions.The Type-P-Finite-One-way-Delay-Poisson/Periodic-Stream samples
from each sub-path are summarized as a histogram with 1 ms bins
representing the one-way delay distribution.From , the distribution of the sum
of independent random variables can be derived using the
relation:This relation can be used to compose a quantile of
interest for the complete path from the sub-path delay
distributions. The histograms with 1 ms bins are discrete
approximations of the delay distributions.Type-P-One-way-Composite-pdv-refmin-NPA for the complete Source
to Destination path can be calculated by combining statistics of
all the constituent sub-paths in the process described in clause 8 and Appendix X.The delay distribution of a sufficiently large stream of packets
measured on each sub-path during the interval [T, Tf] will be
sufficiently stationary and the sub-path distributions themselves
are sufficiently independent, so that summary information describing
the sub-path distributions can be combined to estimate the delay
distribution of complete path.It is assumed that the one-way delay distributions of the
sub-paths and the complete path are continuous.See the common section.In addition to the common deviations, a few additional sources
exist here. For one, very tight distributions with range on the
order of a few milliseconds are not accurately represented by a
histogram with 1 ms bins. This size was chosen assuming an implicit
requirement on accuracy: errors of a few milliseconds are acceptable
when assessing a composed distribution quantile.Also, summary statistics cannot describe the subtleties of an
empirical distribution exactly, especially when the distribution is
very different from a classical form. Any procedure that uses these
statistics alone may incur error.If the delay distributions of the sub-paths are somehow
correlated, then neither of these composition functions will be
reliable estimators of the complete path distribution.In practice, sub-path delay distributions with extreme outliers
have increased the error of the composed metric estimate.See the common section.This metric requires a stream of packets sent from one host
(source) to another host (destination) through intervening networks.
This method could be abused for denial of service attacks directed at
destination and/or the intervening network(s).Administrators of source, destination, and the intervening
network(s) should establish bilateral or multi-lateral agreements
regarding the timing, size, and frequency of collection of sample
metrics. Use of this method in excess of the terms agreed between the
participants may be cause for immediate rejection or discard of
packets or other escalation procedures defined between the affected
parties.Active use of this method generates packets for a sample, rather
than taking samples based on user data, and does not threaten user
data confidentiality. Passive measurement must restrict attention to
the headers of interest. Since user payloads may be temporarily stored
for length analysis, suitable precautions MUST be taken to keep this
information safe and confidential. In most cases, a hashing function
will produce a value suitable for payload comparisons.It may be possible to identify that a certain packet or stream of
packets is part of a sample. With that knowledge at the destination
and/or the intervening networks, it is possible to change the
processing of the packets (e.g. increasing or decreasing delay) that
may distort the measured performance. It may also be possible to
generate additional packets that appear to be part of the sample
metric. These additional packets are likely to perturb the results of
the sample measurement.To discourage the kind of interference mentioned above, packet
interference checks, such as cryptographic hash, may be used.Metrics defined in this memo will be registered in the IANA IPPM
METRICS REGISTRY as described in initial version of the registry .A long time ago, in a galaxy far, far away (Minneapolis), Will Leland
suggested the simple and elegant Type-P-Finite-One-way-Delay concept.
Thanks Will.>>>>>>>>>>>>Issue:Is Section 4.1.8.4 really describing a new error case, about
Alternate Routing? Or does Section 4.1.8.1 on sub-path differences cover
it all?RESOLUTION: The section was re-worded in -10 version to make the
topic, Absence of a real Route between the Src and Dst, more clear.>>>>>>>>>>>>>>>>>>>>>>>>Issue:What is the relationship between the decomposition and composition
metrics? Should we put both kinds in one draft to make up a framework?
The motivation of decomposition is as follows:The One-way measurement can provide result to show what the network
performance between two end hosts is and whether it meets operator
expectations or not. It cannot provide further information to engineers
where and how to improve the performance between the source and the
destination. For instance, if the network performance is not acceptable
in terms of the One-way measurement, in which part of the network the
engineers should put their efforts. This question can to be answered by
decompose the One-way measurement to sub-path measurement to investigate
the performance of different part of the network.Editor’s Questions for clarification: What additional
information would be provided to the decomposition process, beyond the
measurement of the complete path?Is the decomposition described above intended to estimate a metric
for some/all disjoint sub-paths involved in the complete path?>>>>>>>>>>>>>>>>>>RESOLUTION:
treat this topic in a separate memo>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>IssueSection 7 defines a new type of metric, a “combination”
of metrics for one-way delay and packet loss. The purpose of this metric
is to link these two primary metrics in a convenient way.Readers are asked to comment on the efficiency of the combination
metric.>>>>>>>>>>>>>>>>>RESOLUTION:
If a delay singleton is recorded as having "undefined" delay when the
packet does not arrive within the waiting time Tmax, then this
information is sufficient to determine the fraction of lost packets in
the sample, and the additional loss indication of this combo is not
needed.>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
IssueCan we introduce multicast metrics here, without causing too much
confusion? Should the multicast version of this draft wait until the
Unicast concepts are stable (or maybe appear in a separate draft)?>>>>>>>>>>>>>>>>RESOLUTION:
No and Yes.Internet protocol data communication service - IP packet
transfer and availability performance parametersNetwork Performance Objectives for IP-based ServicesIntroduction to the Theory of Statistics, 3rd
Edition,McGraw-Hill NY NY