Internet DRAFT - draft-eronen-rfc-selective-experiment
draft-eronen-rfc-selective-experiment
Network Working Group P. Eronen
Internet-Draft Nokia
Expires: August 31, 2006 J. Arkko
H. Levkowetz
Ericsson
February 27, 2006
Selective Copy-Editing Experiment for RFCs
draft-eronen-rfc-selective-experiment-00.txt
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on August 31, 2006.
Copyright Notice
Copyright (C) The Internet Society (2006).
Abstract
This document proposes an RFC 3933 [RFC3933] experiment for the IETF
RFC publication process. The experiment is limited in scope and
duration. The specific experiment has been chosen because (a) it has
potential to provide a significant process improvement, (b) it can be
executed at a low cost, (c) it addresses a widely recognized problem
in the IETF process, and (d) tool support can be (and has been) built
Eronen, et al. Expires August 31, 2006 [Page 1]
Internet-Draft Selective Copy-Editing Experiment February 2006
for it. The experiment relates to the copyediting and other manual
tasks in the publication process. Specifically, the amount of work
these manual tasks require differs widely between drafts, and for a
certain subset of drafts there are either very minor editorial
changes or no changes needed at all, if we discount the different
formatting requirements between RFCs and drafts. The experiment
involves identification of this subset of drafts and processing them
separately with a "fast path" process that uses almost entirely
automated processes. For the drafts that belong to this subset, it
is expected that the RFC Editor queuing time is reduced from months
or years to weeks or less.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Proposed Process . . . . . . . . . . . . . . . . . . . . . . . 4
3. Process Experiment . . . . . . . . . . . . . . . . . . . . . . 7
4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.1. Incentives . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2. Document Quality . . . . . . . . . . . . . . . . . . . . . 9
4.3. IESG Workload . . . . . . . . . . . . . . . . . . . . . . 9
4.4. Appeals . . . . . . . . . . . . . . . . . . . . . . . . . 10
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
6. Security Considerations . . . . . . . . . . . . . . . . . . . 10
7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10
8. Informative References . . . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
Intellectual Property and Copyright Statements . . . . . . . . . . 13
Eronen, et al. Expires August 31, 2006 [Page 2]
Internet-Draft Selective Copy-Editing Experiment February 2006
1. Introduction
When a document is approved by the IESG, it is sent to the RFC Editor
for publishing. The publication process includes processing at IANA,
copyediting for grammar and spelling purposes, waiting for references
to be published, formatting of the document in the RFC style, final
checks by authors in so called "Author's 48 Hours", and the storage
of the draft and meta-information related to it in the RFC Editor's
information systems.
Unfortunately, the entire process can take a long time, from months
to over a year. There are multiple reasons behind such long
processing delays:
o RFC Editor or IANA workload. This includes the workload of their
subcontractors, such as the professional copyediting services
sometimes employed by the RFC Editor.
o Waiting for normative references to progress through the IETF and
RFC publication processes.
o Busy, unresponsive or hard-to-reach authors that do not respond in
AUTH48 or do not participate in discussions with IANA.
o Events happening in parallel elsewhere that cause changes to be
considered for the documents in question. This leads to longer
AUTH48 periods, going back to the WG or ADs to confirm changes,
and sometimes even pulling the documents back from the queue.
o Imperfect administrative processes, tool support, or lack of input
materials. Documents have been known to fall between the cracks.
The publication process itself is not fully automatic, and
sometimes there RFC Editor has to do extra work when input
material such as XML source is missing.
o Bad document quality which results in more copyediting and
sometimes leads to a need to make even technical modifications.
Overall, the current delays are problematic for the IETF. The
contributors expect to be able to ship products based on RFCs as soon
as the specifications are approved by the IESG. The long wait after
the approval delays the deployment of Internet technology, is not
motivating for the participants, and brings uncertainty that can be
harmful. Long processing times increase the likelihood of events
that prompt people to request "bis" level changes in AUTH48, due to
implementation experience, for instance.
If the entire process for creating new specifications is lengthy, it
Eronen, et al. Expires August 31, 2006 [Page 3]
Internet-Draft Selective Copy-Editing Experiment February 2006
can become a barrier to standardizing new technology. An efficient
IETF process serves the needs of the Internet community best. At a
high level, the process consists of various parts, such as chartering
and processing at WG, IESG, and RFC Editor. All of these parts take
a significant amount of time, and need to be addressed separately if
a more efficient process is desired.
This document proposes a limited experiment to be conducted for the
RFC publication process part, with an expectation of providing a
significant improvement in the processing time for a subset of
drafts. It is expected that the experiment will not cause
significant extra work load either in the IETF or for the RFC Editor;
tool support for this experiment is released simultaneously with the
publication of this proposal.
The experiment relates to the copyediting and other manual tasks in
the RFC publication process. Specifically, it has been observed that
the amount of work these manual tasks require differs widely between
drafts, and that for a certain subset of drafts, there are either
very minor editorial changes or no changes at all, excepting only the
different formatting requirements between RFCs and drafts. The
experiment involves identification of this subset of drafts under
IETF control, and processing them separately with a "fast track"
process that uses almost entirely automated processes.
Under the proposed experimental process the published text is
produced directly from the XML source of the draft that was approved
for publication by the IESG, as supplied by the authors. Under these
conditions, the AUTH48 is also eliminated as being superfluous.
The rest of this document is structured as follows. Section 2
describes the proposed process, Section 3 describes the experiment to
employ it, and Section 4 discusses the some of the implications of
this experiment as well as some potential future enhancements, should
the experiment prove successful.
2. Proposed Process
Under the proposed process, in certain cases the IETF (not the RFC
Editor) makes the decision of how much copyediting the document
should get. The goal of this change is to focus the limited
copyediting resources on those documents which would benefit most
from it, and to allow documents with "good enough" language to
proceed directly to publication.
In order for these documents to benefit from undergoing less
copyediting, the process focuses only on a very restricted subset of
Eronen, et al. Expires August 31, 2006 [Page 4]
Internet-Draft Selective Copy-Editing Experiment February 2006
documents, namely those that do not require any special manual
operations other than copyediting. A document is eligible for the
new process if it fulfills the following conditions when it is
approved for publication by the IESG:
1. It is an IETF document that has gone through normal IETF last
call and undergoes normal IESG evaluation.
2. It has an IANA section indicating there are no IANA actions.
3. It has no Internet Drafts as normative references.
4. It receives no IESG or RFC Editor notes during the IESG process.
5. It has been generated using XML2RFC so that automatic processing
(including re-formatting in RFC style) becomes possible.
The fulfillment of these conditions is either already checked during
the existing process or it can be determined programmatically. Our
preliminary analysis suggests that more than 20% of all current
Working Group drafts meet these criteria.
An eligible document is included in this experiment if the IESG
decides that copyediting would not significantly improve the quality
of the document. This determination of having "good enough" language
is a human decision, made by the IESG during the course of their
technical review. Since the area directors often read the drafts for
the first time during IESG evaluation, they will get a fairly good
impression of how difficult the document is to read for someone who
has not seen it before. Thus, we believe the IESG is a good place to
make this decision. Furthermore, given that the IESG has to read the
document in any case, this check does not represent a major increase
of workload for the IESG. Note that the IESG will only record their
impression of the language quality. It would not be a good use of
the IESG's time to ask them to write down the observed problems or
suggest improvements.
Under the new process, IESG members voting "Yes" for a given document
MUST provide and others MAY provide an indication of whether the
document has sufficiently good language. When providing this
indication, the IESG members should consider the following aspects:
o Does the document contain more than a minor number of typos,
grammar mistakes, or unnecessarily difficult-to-understand
language?
o Would copyediting, done by a person who is not familiar with the
technical content, significantly improve the document? Or does
Eronen, et al. Expires August 31, 2006 [Page 5]
Internet-Draft Selective Copy-Editing Experiment February 2006
the problems lie in aspects such as overall document organization,
or bad protocol design, that cannot be realistically be improved
by such copyediting?
o What is the purpose of the document? For instance, fine-tuning
the language would less important for a document documenting a
design discussion for posterity, and more important for a protocol
specification that will be read by a large number of people
implementing and using the protocol.
o What is the intended status (proposed standard, informational,
etc.) of the document?
An eligible document that has received solely "good enough"
indications from the IESG is chosen for the fast track process. The
RFC Editor determines eligibility according to items 1. to 5. of
Section 2, using a tool, and inspects the IESG language quality
decision. The fast track process, if chosen, eliminates the IANA,
copyediting, reference wait, manual format conversion, and AUTH48
steps. The following steps are followed:
IESG note to the RFC Editor
A special IESG Note to the RFC Editor is attached to the approval
decision. This note indicates to the RFC Editor what level of
copyediting is believed to be necessary.
Acquire XML source
As is already done currently, the RFC Editor asks the authors of
recently approved RFCs for the XML source of the draft. If such
source is not available, the regular process is followed.
Verify XML source correctness
The RFC Editor verifies that they have been given the correct XML
source by running it through XML and verifying that there are no
differences in the document text (for instance by running 'diff').
If the source is incorrect, go back to the previous step.
Determine eligibility
The RFC Editor determines the eligibility of the draft, for
instance by employing a new tool to check reference status and
other requirements necessary for the new process.
Eronen, et al. Expires August 31, 2006 [Page 6]
Internet-Draft Selective Copy-Editing Experiment February 2006
Format conversion
There exists an option which can be given to a new version of the
XML2RFC tool which enables it to generate output suitable for an
RFC. XML2RFC with this option set is used to generate the final
RFC output.
Storage
The RFC Editor updates their information systems (such as the
database of RFCs) with the new RFC and its metadata, and makes the
new RFC publicly available from the RFC repository.
This process is done at the document approval notice reception /
author response reception time. The RFC Editor often acts
immediately after the approval notice is received, so the fast track
process has the potential of resulting in a published RFC within days
of the approval.
The effect of possible appeals of the IESG's decision to publish a
document on this process is covered in Section 4.4.
3. Process Experiment
We believe the crucial question to answer is "Does the publication
process work better with the modification proposed in this document,
or without it?"
If the answer to this question is found to be "yes", then this change
should be done, independently of other, more ambitious projects such
as determining the overall requirements for technical publication
services [TechPub]. However, an experiment is needed to better
evaluate the effects of the proposed process.
The experiment needs to have a limited scope and duration. The scope
of the experiment is naturally limited by the eligibility rules, so
it is suggested that for a duration of one year, all drafts
satisfying the eligibility and language quality rules will be run
through the new process. The experiment is set to begin at the time
this document is approved for publication.
Based on our initial analysis, we expect that roughly 100 documents
could be eligible based on the checklist in Section 2; depending on
the quality of the language in these documents, we expect somewhere
between 25 and 50 documents to use the fast track process. The
figure depends highly on how strong incentives this experiment
creates for the authors to improve the language before IESG approval.
Eronen, et al. Expires August 31, 2006 [Page 7]
Internet-Draft Selective Copy-Editing Experiment February 2006
During the experiment, the RFC Editor collects separate statistics of
the processing and queuing times for the regular and fast track
processes. A person designated as the experiment supervisor collects
feedback from document authors and WG chairs (using feedback forms)
for documents going through the new process. At the end of the
experiment feedback is also solicited from the IESG.
An evaluation is performed at the end of the experiment and the
results are published as an Internet Draft. The evaluation involves
employing the collected statistics to determine the effect of the
fast track process on document processing time and effort at the RFC
Editor organization, and evaluating the summary of received feedback.
4. Discussion
This change allows the IESG to focus the limited copyediting
resources to documents where the benefits are the largest. It is
expected that the reduced workload for the subset of documents that
go through this process also will reduce the processing time of other
documents, given the same level of resources devoted to the RFC
Editor activities.
In making the copyediting level decision, the IESG is in a better
position than the RFC editor to consider all the factors involved
(e.g., purpose of the document, its priority, etc.). We believe it
is in the interest of IETF community to make this decision
transparently. When this decision is recorded in the public records
that the IESG makes available, this condition is fulfilled.
A more fine-grained scheme that goes beyond on/off control of the
copyediting can also be considered. However, it is suggested that
such a scheme be considered only if the process described in this
document prove useful.
We have not yet fully analyzed what other choices exist for making
this decision than IESG evaluation. Some other alternatives appear
to be possible as well. For instance, there are a number of review
boards such as GEN-ART that could also provide input.
4.1. Incentives
We believe this arrangement would better align the incentives of
various parties with the IETF's goals.
Specifically, this process ensures that the IETF has control over the
level of copy editing. If the RFC Editor function is contracted to a
for-profit entity, that entity has an incentive to increase the
Eronen, et al. Expires August 31, 2006 [Page 8]
Internet-Draft Selective Copy-Editing Experiment February 2006
amount of copyediting, and ask for more funding. In a true open
market situation, competition between service providers would control
this, but we do not currently believe there will be significant price
competition for the RFC Editor contract.
The experiment also provides better incentives for document authors,
WG chairs, and other reviewers. If better-quality text means that
the document progresses faster, people interested in the document
have an incentive to fix the text earlier (e.g., provide more
editorial comments during WG last call). These people are also in
good position to know which changes are purely editorial and which
would actually change the meaning of the text.
4.2. Document Quality
One argument that could be made against this experiment is that less
involvement by the RFC editor means that quality will suffer.
We believe the experiment will have exactly the opposite effect. The
editing work done by the RFC editor does very little to increase the
quality of documents that are already in a pretty good shape. This
experiment allows focusing the limited resources on those documents
where the "return on investment" is the largest. It also creates
incentives for the authors to work on the language before the
document is approved.
Another potential complication is the difference between the XML2RFC
output and the style currently used by the RFC editor. Discussions
about the exact formatting requirements have been going on since
November 2005, and when used with the "rfcedstyle" option, version
1.31pre4 produces output that is believed to match the current RFC
editor style. While it is possible that some differences remain, and
that the preferred style changes over time, we believe the current
formatting is more than acceptable, and fine-tuning it further, while
possibly beneficial, will not produce a substantial added value for
the IETF community.
4.3. IESG Workload
The experiment intentionally proposes a very simple process for
determining which documents meet the language quality criterion, as
explained in Section 2.
We expect that the IESG members can make this decision without
spending any more time than they already do. We also do not expect
the IESG to produce an explanation of why the document was or was not
chosen, list any of the language problems identified in the document,
or to negotiate about the decision with the document authors.
Eronen, et al. Expires August 31, 2006 [Page 9]
Internet-Draft Selective Copy-Editing Experiment February 2006
4.4. Appeals
One possible problem with the experiment is that [RFC2026] specifies
a two-month period for appealing IESG decisions. Some people have
interpreted this to mean that no document can be published faster
than this.
However, appeals to the IESG are almost always filed within days of
the decision. Even in cases when writing the complete appeal text
may take some time, a "notice of intention to appeal" is often given
immediately. The "fast track" process is also limited to documents
that go through IETF Last Call, and people who appeal the IESG
decision read and comment the documents already during the last call.
Furthermore, the two-month period has not been observed rigidly
recently. While the average RFC queue delay ensures that plenty of
time is left for appeals, in 2005 five RFCs were published less than
two months from their approval. (We would be interested in knowing
how this was possible, so we could try to get our documents to this
"fast track" -- the record processing time was only five days! :-).
Moreover, in the case of standards track and BCP RFCs, the IESG can
always reverse its decision after the RFC has been published by
downgrading the document to "Historic".
For this experiment, we suggest that documents known to be
controversial should not be selected for the fast track process.
Since more than 99% of documents are not appealed, this is unlikely
to affect the number of documents in the experiment. In addition, it
is suggested that for the documents selected for the fast track, the
approval notices include a request to make the intent to appeal known
within two weeks.
5. IANA Considerations
This document has no IANA Actions.
6. Security Considerations
This document does not introduce any new security considerations.
7. Acknowledgments
The authors would like to thank Bob Braden, Brian Carpenter, and
Stephen Hayes for ideas and interesting discussions about this
Eronen, et al. Expires August 31, 2006 [Page 10]
Internet-Draft Selective Copy-Editing Experiment February 2006
problem space.
8. Informative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision
3", RFC 2026, October 1996.
[RFC3933] Klensin, J. and S. Dawkins, "A Model for IETF Process
Experiments", BCP 93, RFC 3933, November 2004.
[TechPub] Mankin, A. and S. Hayes, "Requirements for IETF Technical
Publication Service", draft-mankin-pub-req-01 (work in
progress), October 2005.
Eronen, et al. Expires August 31, 2006 [Page 11]
Internet-Draft Selective Copy-Editing Experiment February 2006
Authors' Addresses
Pasi Eronen
Nokia Research Center
P.O. Box 407
FIN-00045 Nokia Group
Finland
Email: pasi.eronen@nokia.com
Jari Arkko
Ericsson
FI-02420 Jorvas
Finland
Email: jari.arkko@ericsson.com
Henrik Levkowetz
Ericsson
Torsgatan 71
111 37 Stockholm
Sweden
Email: henrik@levkowetz.com
Eronen, et al. Expires August 31, 2006 [Page 12]
Internet-Draft Selective Copy-Editing Experiment February 2006
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2006). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Eronen, et al. Expires August 31, 2006 [Page 13]
