An assessment regarding the Ninth SOSP Submissions -or- How (and just how Not) to publish a great systems Paper
Roy Levin and David D. Redell, Ninth SOSP Program Committee Co-chairmen
Introduction
On March 21, 1983, this program committee for the 9th Symposium on running System Principles, having browse the eighty-three documents submitted, chosen sixteen for presentation in the symposium. This acceptance ratio of approximately one in five approximates those of previous SOSPs, even though true quantity of submissions had been significantly less than in the last few years. A few users of this program committee discovered it interestingly simple to split up the good documents from the bad ones; indeed, the ten committee users quickly agreed upon the disposition of over 80% regarding the documents. Once the acceptance ratio shows, many of these had been rejections.
Following the committee had completed its selectio n process, a few people indicated disappointment into the overall quality associated with submissions. Lots of the refused documents exhibited similar weaknesses, weaknesses that the committee felt must have been obvious into the writers. Within the hope of increasing the standard of future SOSP submissions, and systems documents generally speaking, the committee chose to explain the requirements found in assessing the documents it received. This informative article combines the requirements utilized by most of the users of the committee, not merely the authors.
To attempt to avoid sounding preachy or pedagogic, we’ve cast this presentation in the 1st and 2nd person and adopted a light, periodically funny design. Nonetheless, the intent is severe: to indicate the typical conditions that look over and over repeatedly in technical papers in a fashion that will likely make it easier for future authors to prevent them. Yourself to be a prospective author for the 10th SOSP or for TOCS as you read this article, then, suppose. You have done some ongoing work you want to publish, so that you take a seat to create a paper. Just just What concerns for anyone who is thinking about as you compose? They are additionally the concerns that people, the reviewers of one’s paper, are going to be asking to find out its suitability for book.
Classes of documents
Your paper will probably fall obviously into certainly one of three groups:
- It presents a real system, either by a worldwide study of a complete system or by a selective study of particular themes embodied in the machine.
- It presents a method this is certainly unimplemented but uses ideas or methods which you have the community that is technical understand.
- It addresses a subject within the theoretical areas, for instance, performance modelling or safety verification.
Clearly, a solitary collection of assessment requirements may not be used uniformly across these categories; however, many requirements use equally well to all the three. We will try to emphasize the classes of papers to which it applies as we describe each one below. Usually it will be obvious from context.
Requirements for Evaluation of Submissions
Original A Few Ideas
Would be the basic a few ideas when you look at the paper brand brand new? There’s no point in publishing a paper up to a meeting or log worried about initial work unless the paper contains one or more brand new concept.
How will you understand? You should be acquainted with the state associated with the art and research that is current the location covered by your paper so that you can understand that your projects is initial. Probably the many typical failing among the submissions in the 1st category (genuine systems) ended up being an lack of the latest tips; the systems described were often isomorphic to 1 of only a few pioneering systems well-documented when you look at the literary works.
Are you able to state the new concept concisely? If the paper is always to advance their state of real information, your audience must certanly be capable of finding the brand new some ideas and comprehend them. Take to composing each idea down in a paragraph that some body generally speaking versed when you look at the area that is relevant comprehend. If you fail to, look at the possibility that you do not actually comprehend the concept your self. Them in the abstract for the paper when you have the paragraphs, use.
Precisely what is the nagging issue being fixed? Your audience may not be anticipated to imagine the nagging issue you faced given just a description for the solution. Be certain. Make sure to explain why your trouble could not be resolved just as well by formerly posted strategies.
Will be the tips significant adequate to justify a paper? Usually, documents explaining genuine systems have a couple of tiny improvements of founded methods. The brand new idea(s) may be described in a few paragraphs; a twenty-page paper is unneeded and frequently obscures the innovation that is actual. The author of the paper sometimes unconsciously confuses the total effort with the work that is actually new since construction of a real system is a lot of work. (“My group labored on this technique for just two years and we’re finally done. Let us tell the global globe exactly just how wonderful it really is.”) In the event that innovation is tiny, a little paper or technical note in an appropriate log is right than an SOSP submission.
May be the ongoing work described notably different from existing related work? an extension that is obvious a formerly posted algorithm, method, or system, doesn’t generally justify publication. Needless to say, the label “obvious” must certanly be used carefully. ( recall the tale of Columbus showing steps to make an egg stand at a stretch (by carefully crushing it): “it’s apparent once i have shown you how”.) You have to show that the work represents an important departure from their state regarding the art. If you cannot, you ought to think about why you might be composing the paper and exactly why anybody except your mother should would you like to see clearly.
Is all related work referenced, while having you actually see the cited material? You should have trouble convincing the skeptical audience of this originality of one’s efforts from previously published work unless you specifically distinguish it. This calls for citation. Additionally, you’ll find it harder to persuade your audience for the superiority of one’s approach you haven’t if he has read the cited works and.
Are evaluations with past work explicit and clear? You can’t simply state: “Our approach varies notably from that used when you look at the BagOfBits system 3.” Be particular: ” Our memory that is virtual management makes use of magnetic news in the place of punched paper tape as with the BagOfBits system 3, because of the expected improvements in transfer price and janitorial expenses.”
Does the task comprise an extension that is significant validation, or repudiation of previous but unverified tips? Execution experiences supporting or contradicting a formerly posted paper design are really valuable and worthy applicants for book. Designs are inexpensive, but implementations (specially those centered on unsound designs) are costly.
What’s the earliest paper you referenced? The modern? Have actually you referenced work that is similar another organization? Have actually you referenced technical reports, unpublished memoranda, individual communications? The answers to these concerns help alert you to definitely blind spots in your knowledge or understanding. Usually, papers with just venerable recommendations repeat recently posted work of that the writer is unaware. Papers with only current references often “rediscover” (through ignorance) old tips. Documents that cite just unpublished or material that is unrefereed to have problems with narrowness and parochialism. Understand that citations not merely acknowledge a financial obligation to other people, but additionally act as an abbreviation device to spare your audience a development that is complete very very first axioms. In the event that audience has to get some good of the development, nonetheless, he must certanly be in a position to transform your citations into supply product he is able to read. Private communications and memoranda that are internal this test. Technical reports are generally posted in restricted amounts, out-of-print, and hard to get. Consequently, such citations as supply product should really be avoided whenever we can.
In the event that system is implemented, exactly exactly exactly how has it been utilized, and what has this usage shown in regards to the importance that is practical of a few ideas? Yet again, a numerous implementation that is man-year doesn’t of itself justify book of a paper. In the event that implemented system contains new tips, it is vital to explain the way they worked call at training. a apparently good notion that don’t pan away is at minimum since interesting as you that did. You will need to be certain and accurate. “Our climate prediction system is ready to go with no you’ve got reported about its periodic inaccurate forecasts” is a lot less convincing than “everytime we are not able to forecast rainfall, the users hang their damp shirts within the tape drives research topics for psychology to dry”. Into the second instance, at minimum we all know that folks are utilizing and with respect to the system.