Test Traffic Measurements (TTM)
as a Membership Service
Henk Uijterwaal (firstname.lastname@example.org)
October 23, 2000
This document describes the model for selling Test Traffic
Measurements (TTM) as a membership service. This document is based on
discussions at the various RIPE-meetings as well as input from the RIPE NCC
The actual sale of test-boxes as well as the services provided by the RIPE NCC
will be subject to the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement''. These will be published as a separate
document. In case of any differences between this document and the
``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'', the latter document will take preference.
Since early 1997, the RIPE NCC has been working on the Test-Traffic
Measurements project (TTM). The goal of this project is to do independent
measurements of Inter-Provider network parameters, such as One-way-Delay,
Packet-Loss or Routing vectors, as specified in RFC's 2330, 2679 and
2680 . The design goals have been described in an earlier
document . Several papers about the implementation and
first results of the project have meanwhile been published .
The RIPE NCC has always used a model where new, experimental projects were
funded from the general budget. However, once a project becomes more
mature and can be offered as a regular service to its membership, the
costs for the service are charged to the sites using the service.
So-far, the TTM project was funded from the general budget, as, when the
project started, it was not clear if the project proposed in the design
document  could actually be implemented. This situation
has changed by now, the RIPE NCC has proven over the last years that it
can routinely do delay- and loss-measurements on a large scale.
A direct consequence of the funding model was that participation in the
project was limited to current RIPE NCC association members. After all,
they funded the project. Also, as the number of ``test-boxes''
(measurement stations) was limited, we only installed 1 test-box at each
member in order to give as many sites as possible a chance to participate.
Over the last years, we have seen that these limitations are too
restrictive. First of all, a fair number of sites which are currently not
RIPE NCC members are interested in hosting one or more test-boxes. These
sites usually do not need the LIR services offered by the RIPE NCC,
because they either are not LIR's or already receive these services from
Then, several large customers with more than 1 PoP have expressed interest
in hosting more than one test-box in order to include their internal
networks in the measurement chain as well, or to provide their customers
with measurement data between each PoP and the outside world.
For all these reasons, the RIPE NCC, in consultation with its
AGM , has decided to move the TTM from an experimental project
to a service that will be offered to the entire community.
This document describes the model for offering TTM as a service. In the
first half (section 2) the principles behind the operation are
described; the second half (section 3 onwards) describes the
implementation details as well as the service agreement between the
RIPE NCC and the site participating in the project.
Sale of the test-boxes as well as the service offered by the RIPE NCC,
will be subject to the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'' . In the following
sections, we will summarize that document, as well as give an extensive
overview of what is being offered as part of the TTM service.
This document was accurate and complete when it was published. However,
the ``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement''  may have been changed since then and what
is currently included in the TTM service may have changed as well.
Before you buy a test-box, you will receive a copy of  for
you to review and sign.
In case of differences between this document and the copy
of  signed by you, the latter document will take preference.
The basic model for participating in the TTM can be described in 2 paragraphs:
- First of all, the site interested in participating in the service
orders one or more test-boxes from the RIPE NCC. These test-boxes will
be installed at the site of this organization. The host site pays for
the hardware of the test-boxes and the test-boxes become the property
of the host site.
- In parallel, the organization pays a yearly service fee for the
operation of the test-boxes. This service fee covers the cost of
operation of the test-box by the RIPE NCC, analysis of the data,
software updates, access to
all products based on the TTM-data and everything else needed to keep
the box running. Annually, the host site
can decide to renew the service contract.
Sale of the test-boxes as well as the service will be subject to the
``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'' .
A test-box currently costs approximately Euro2,500.- ,
installation costs another Euro250.- for a total of Euro2,750.-.
The installation and shipping costs will be reduced if a site orders more
than one test-box in a single order. Shipping, local taxes, import duties,
and such will have to be paid by the host site.
The test-boxes have a 1 year warranty. The RIPE NCC will buy a few spare
test-boxes. In case of a hardware failure, the box will either be repaired
or exchanged with a similar test-box from the spares, the latter only if
this is possible without having to re-import the test-box.
More details can be found in section 3.
No service fee will be charged in the year 2000. The AGM  has
asked that the TTM service should generate a self-sufficient budget in
2001. Our first estimates show that this implies that the service should
generate approximately Euro300,000 in service fees in that year.
Assuming that we increase the measurement network to about 100 nodes in
2000, this implies a service fee per test-box of Euro3,000. Volume
discounts for sites hosting several test-boxes will be discussed at a
The service fees are intended to recover those costs which the RIPE NCC
incurs by providing the TTM service. These are still somewhat hard to
predict exactly and adjustments will be made in the coming years according
to experience. The expectation is that the service fees will drop as a
consequence of more participants sharing the total cost.
At the moment, it is not foreseen that the service fee will include
membership of the RIPE NCC association. However, recent developments have
indicated that offering a service without membership may be in conflict
with the by-laws or tax-status of the RIPE NCC. This is being
As the RIPE NCC and the RIPE NCC board do not want to lose momentum
with the introduction of this service, it has been decided to set the
service fee for 2001 to Euro0 for the time being. However, any new
sites will have to sign the service contract to allow for the introduction
of a service fee in the future.
The service fee covers operation of the test-box, access to the data
according to the current data-disclosure policy  and
products based on the TTM data as well as maintenance and support, as
discussed in more detail in section 4.
The RIPE NCC proposes that all sites, including the ones currently hosting
a test-box, will pay the service fee when this is introduced. However,
any hardware that has already been installed will be given to those sites
for free. Existing sites will be asked to sign the service contract in
order to create a legally binding agreement between the RIPE NCC and the
The RIPE NCC is aware that not all sites currently participating in the
TTM, have budgeted resources for this service for 2001 and is prepared to
discuss different transition plans for those sites.
There is no warranty on the currently installed hardware anymore. If
anything breaks, the box can only be fixed against cost. The next
generation of test-boxes will have clocks that are easier to operate and
maintain. Once the RIPE NCC has some running experience with these
clocks, it will offer the possibility to upgrade the existing (Motorola)
clocks to the newer (Trimble) clocks against cost.
So-far, all measurements done with the test-boxes are what we call
``public experiments'', that is, the test-box is operated as a black box
by the RIPE NCC, measurements are being done to other test-boxes hosted by
other sites and the data is made available to both the hosts of the
sending and receiving test-boxes.
However, there has been some interested in doing ``private'' experiments
with the test-boxes, for example:
Although the RIPE NCC wants to focus on public experiments, it does want
to help sites to set up their private experiments. For this reason,
test-boxes will also be sold without a service contract. The details of
this agreement will be discussed with the potential customers at a later
- Measurements on the internal network, where the host site does not
want to share the data with others,
- Other experiments that can be done with the test-boxes, or
- Measurements on experimental networks that are not connected to the
In order to guarantee the integrity of the data, a test-box cannot be used for both public and private
experiments at the same time.
In this paragraph, we have assumed that the number of requests for
test-boxes for private experiments is small. If the number grows, we may
have to revisit this model.
Several sites have asked if it would be possible to build their own
test-boxes instead of ordering them from the RIPE NCC. In some countries
this might be cheaper, though one should keep in mind that the RIPE NCC
orders a large number of test-boxes at the same time and thus gets volume
Setting up a test-box will also require more work (of the order of a day)
and knowledge (in particular of the FreeBSD Operating System) at the
The advantage is that setting up a test-box will sometimes be faster, as
one does not have to wait for the RIPE NCC to produce a test-box.
If, in the future, it turns out that there is sufficient interest in this,
the RIPE NCC will publish the requirements for a machine to be used as a
test-box. Note that only a very small number of hardware configurations
can be tested by the RIPE NCC. Any hardware not on the list will
not be supported.
If the test-box will be used for ``public'' experiments, RIPE NCC will
provide the host with instructions on how to configure the box. After the
box has been set up, the host site should transfer control of the machine
to the RIPE NCC. The host site should also sign a service
agreement  for the test-box and pay the service fee. After
these formalities have been taken care of, the RIPE NCC will then download
the necessary software to the machine and start doing measurements.
The service contract gives the site hosting the test-box the right to use
the TTM software on the test-box, to review the source code to detect any
possible security risks that it may introduce at the hosting site and a
few other basic rights according to Dutch law. The license to use the
software expires when the service contract ends. Any other usage of the
TTM software is not allowed.
However, the software can be licensed to be used for other purposes. For
this case, we propose to use the GPL . Details will be
discussed in a future document.
The TTM service will continue to be improved and expanded. The global
development plans as well as the service fees for the next year will be
described in the annual RIPE NCC Activity Plan. This activity plan is
discussed and approved by the AGM of the RIPE NCC.
On a less formal level, the RIPE NCC will present its development plans
for the TTM service in the RIPE Test Traffic-Working Group (TT-WG) meetings
and mailing-list, where they can be discussed. These meetings and mailing
list are open to everybody interested in this service.
Test-boxes can be ordered by filling out the online order-form with the
necessary configuration data. After that, you will receive a copy of the
service contract. This has to be signed by somebody authorized to do so in
your organization. The RIPE NCC will also send an invoice with
instructions on how to pay for the box.
We will assume that you have read the installation
document  before ordering a test-box and that you have
checked that your site meets the requirements necessary to install and
operate a test-box. We also assume that installation of a test-box will
not violate any local security or safety requirements.
After your order has been received, a test-box will be configured according
to the current production schedule and tested. It will be shipped to your
site as soon as the machine is ready and the payment has been received.
Included in the shipment are:
Not included with the test-box are:
- 1 test-box according to the current specifications.
- 3 year warranty.
- Pre-configured and tested at the RIPE NCC.
- Installation support by email: the mailbox will be checked at regular
intervals during working hours, with the initial response within 1
working day. Installation support includes: getting the test-box
connected to the Internet, help with finding a suitable spot for the
antenna and checking the configuration.
- A keyboard, monitor or mouse. These are not needed for normal
operations but might be useful for troubleshooting. We assume that
the host site can provide one for a short period during testing.
- Taxes, import/export duties, etc. These will be charged to the
recipient and may have to be paid before the test-box is delivered.
- Any special or additional hardware (re-radiators, antenna splitters,
that might be necessary to operate the test-box under circumstances
other than those described in . The RIPE NCC
has experience with test-box operations under special
circumstances and can give advise here. However, any additional
hardware needed in these cases will have to be paid by the host.
- Setting up other hardware such as routers or fire-walls needed to
operate the test-box.
- Shipping to your site, see the next section for details.
- Installation of cables and such.
- On-site installation support by a RIPE NCC engineer.
By default, the test-box will be shipped to the host site by SkyNet. The
box will be insured for the replacement value of the test-box. The current
rates for shipping the boxes can be found in table 1. These
rates are an indication only and may change without notice, please contact
the RIPE NCC for the latest rates.
As an alternative, the box can be sent to the host site by FedEx, UPS or
another company with the costs directly charged to the customer's account.
To use this option, please send us your account details.
| 43 +
|Other European Union Countries
|Other European Countries
|Middle East, Africa
|Former Soviet Union
Table 1: Costs of shipping a test-box to various
destinations. The weight of a test-box with cables and such is between 20
and 25kg, the base rate is for a box of 20kg, x is the number of kilos
above 20. These rates are the upper limit for shipping to a particular
destination, in practice they might be lower. These rates do not
include Dutch Sales Tax (of, at the moment, 17.5%). These rates are an
indication only and may change without notice.
The RIPE NCC has some possibilities to send an engineer to do either a
site-survey or help with the installation of the test-box at a site.
However, all travel and related costs as well as an hourly fee will be
charged to the host site.
The RIPE NCC cannot do electrical safety tests and such. If those are
required, please contact a local expert.
The service fee includes operation of the test-box as a ``black box'' by
the RIPE NCC. Under normal circumstances, the test-box host will not have
to do anything to keep the test-box running. We do, however, expect that
the site appoints a local contact person to help with problems that cannot
be solved remotely. (For example, pressing the reboot button.)
According to , the RIPE NCC will operate the test-box for
the hosting site, collect and analyze the data, and maintain the software,
without giving any details on how this is implemented. This is done for
practical reasons, if the services are specified in detail, any change
would require that an updated contract is signed by both the hosting site
and the RIPE NCC. This would severely limit the RIPE NCC's flexibility in
responding to its customers' wishes.
At the moment, the service fee would include the following:
- Delay/Loss measurements:
- To all sites considered interesting by the RIPE NCC. Hosts can
suggest to include/exclude specific paths and/or change rates.
However, the final decision will be made by the RIPE NCC.
- The latest version of the measurement software.
- Collection of data by the RIPE NCC once a day. All services and
products will be based on data that could be collected by the
central computer at the RIPE NCC between 2:00 and 4:00 GMT. If data
could not be collected for a particular day, the RIPE NCC will try to
collect the data from the test-box the next day until the files
expire after 30 days. All data will be stored, but analysis will
only be done retroactively for files not older than 7 days.
- Services, currently (please see  for an
overview of the current products and services):
- The raw data, in ROOT-format , will be put on our
ftp-site on request. This includes a ROOT software library
necessary to read the data.
- Daily, Weekly, Monthly plots of delays and losses.
- Delay alarms to a specified email address.
- Network Performance Scores when this service becomes available.
(Planned for late 2000.)
- Overview plots for planning purposes when this service becomes
available in 2000.
- All other products based on the TTM data.
- Maintenance of the software.
- Software updates when they become available.
- Automated checks of the data collection processes every 3 hours
and automated recovery/restarts as far as technically possible.
- Checks of log files once every working day, plus feedback from us
to the test-box host in case of problems that require action from
(Data Quality Monitoring, everything else that involves human
- Maintenance of the hardware.
- CPU: Trouble shooting by the RIPE NCC together with the local
test-box contact person. The RIPE NCC will decide if a test-box
has to be sent back for repair.
- Disks: will be replaced as long as they are under warranty.
- Clock: the warranty on the clocks will be transferred to the
- Email service to questions regarding test-box operations, with an
initial response within 1 working day (24h).
Not included are:
- An account on the machine.
- Hardware or software changes are not allowed unless agreed
beforehand with the RIPE NCC.
- All repairs not covered by the warranty as well as shipping costs
when a test-box has to be shipped back for repair. All costs will
be charged to the host site.
Also, the service contract assumes that the host agrees with the
data-disclosure policy. The service contract covers a period of 1 year,
there is no obligation to use the service the next year.
The host site agrees that traffic from/to the test-box will not receive any
special treatment in order to artificially improve the results of the TTM
by that site. The RIPE NCC reserves the right to verify this or to stop
providing the service if it believes that the site violates this agreement.
The host site also agrees to keep all contact information for the test-box
up to date and inform the RIPE NCC in advance if a test-box has to be moved
or the configuration has to be changed.
This list may be changed at any time, please contact us to see if there
are any updates. The RIPE NCC is also always interested in expanding the
TTM service, so any suggestions to extend measurements or analysis are
more than welcome.
- V. Paxson et al,
``Framework for IP Performance Metrics'',
RFC2330, May 1998.
G. Almes et al,
``A One-way Packet Delay Metric for IPPM'',
RFC2679, September 1999.
G. Almes et al,
``A One-way Packet Loss Metric for IPPM'',
RFC2680, September 1999.
- O. Kolkman, H. Uijterwaal,
``Internet Delay Measurements using Test Traffic'',
ripe-158, May 1997, see:
- Henk Uijterwaal et al,
``Internet Delay Measurements using Test Traffic:
First Results'', in Proceedings of the SANE98 Conference,
Maastricht, November 1998.
- Jacqueline M. B. Seignette et al,
``General Terms to the RIPE NCC Test Traffic Measurements Service and Test-Box Purchase Agreement'', ripe-214, September 2000, see:
- ``RIPE NCC Annual General Meeting Minutes 1999 (AGM'99)'',
- Henk Uijterwaal,
``Internet Delay Measurements using Test Traffic,
Data Disclosure Policy'', ripe-180, see:
- GNU General Public License, see
- Henk Uijterwaal,
``Internet Delay Measurements using Test Traffic,
Installing and Hosting a Test Box'', ripe-168, see:
This document will be replaced shortly, check
our website for the current document.
- René Wilhelm, Minutes of the TT-WG Meeting at RIPE35, see
- Rene Brun and Fons Rademakers,
``ROOT - An Object Oriented Data Analysis Framework'',
Proceedings AIHENP'96 Workshop, Lausanne, September 1996,
Nucl. Inst. & Meth. in Phys. Res. A 389 (1997) 81-86.
See also http://root.cern.ch/
as of October 1, 2000, please refer to
for the current prices.
- For example, a
private experiment with a lot of I/O, will affect the results of the
regular measurements. This is hard to detect and correction afterwards is