Research Paper Advice

The PEPM Symposium/Workshop series aims to bring together researchers and practitioners working in the broad area of program transformation and generation. We hope that PEPM authors will find the following advice helpful in preparing their submissions. The suggestions are based on the guidelines for PEPM 2012 submissions. Tool demonstration papers have special requirements, described in ToolPaperAdvice.

Program Goals

The topics covered by PEPM are listed in the Call for Papers. Please contact one of the program chairs if you have further questions about PEPM's scope.

Original Work

All papers should be original work, and not have been previously published nor have been submitted to, or be in consideration for, any journal, book, conference, or workshop. The novelty requirement is relaxed for Tool Demonstration Papers.

Commendable Qualities

• Clear contributions: The contributions of the paper should be stated explicitly in the introduction of the paper and elaborated in the rest of it. Limitations should be acknowledged; if possible, future work should mention the ways to overcome these limitations.
• Well-written: In particular, the authors should strive to make the paper free from spelling and grammatical mistakes.
• Good examples: Every major point should be illustrated with an example.
• Well-supported claims: Any claims of performance improvements, ease of use, scalability, etc. must be justified by the appropriate evidence, such as computational cost analyses, benchmarks, usability studies, multiple case studies and so on.
• Relevant to real-world problems: Authors should clearly indicate actual real-world scenarios in which they envision their work (when fully mature) being applied. Issues such as scalability, analysis precision, degree of automation, learning curve, etc. that might hinder the use of the technique in practice should be acknowledged and assessed.
• Properly positioned to related work: Submissions should include a related work section that summarizes and contrasts with closely related work. Related work should not simply list previous papers with a brief summary of their contents, nor should it state only the strengths of the submission and the weakness of previous work. A proper related work section should state both the strengths and weaknesses of the submission and indicate situations in which one method might be preferred over another.
• Clear methodology and integration into larger development context: Authors should clearly indicate the steps that users should go through to apply a technique including any manual preprocessing, tool configuration, assessment of output, etc. Moreover, an assessment of how the technique fits into the context of other development tools such as debuggers, test frameworks, integrated development environments, etc. For example, if the proposed technique is a generative programming technique in which users are not expected to read generated code, what mechanisms might be provided to aid debugging of generated code. In another case, how does the transformation interact with the use of library code? Is there any impact on how the system is tested?
• Supporting material: If at all possible, the authors are encouraged to make examples, data, tools or other artifacts presented in the paper freely available online.

Common Flaws

• Lack of simple examples: Every major point in the paper should be illustrated with an example. When introducing a new technique or analysis, it is wholly appropriate to use simple, well-known examples such as the "power function" and the "dot product".
• Lack of realistic examples: The paper will hopefully claim that its technique and analyses scale to real-world problems. These claims should also be illustrated with an appropriate large example. If the example is too large to fully describe in the paper, the paper should give the highlights and refer to the complete explanation online: a web page, a technical report, or well-commented source code.
• Lack of clear correctness notions: the authors should explicitly state what it means for their technique or analysis to be correct. For example, a paper on program transformation must state what properties of programs are preserved by the transformation. The authors should argue, perhaps informally, how their implementation or tool meets or approximates the correctness properties, or give a clear statement of the pre-conditions for the successful application of the technique. Detailed proofs need not be included in the submission but may be referred to in a URL or a technical report.
• Lack of clear methodology: Many papers present techniques without telling what the users have to do, step-by-step, to apply the techniques. Issues such as initial binding-time improvement or refactoring of source code, stubbing out or annotating library methods, etc. are often ignored. As a classic example, many early PEPM papers "swept under the rug" the need for manual binding-time improvements and annotations. As the result, non-experts have had significant trouble applying partial evaluation techniques in practice.
• Poor related work comparisons: Papers often suffer from poor related work comparisons in which authors follow a pattern "our tool is better at TASK-X than tool T1 because ..., our tool is better at TASK-Y than tool T2 because ..." (all the while ignoring the fact that tool T1 has a much broader scope than the authors' tool or that it is in fact better than the authors' tool for many aspects not mentioned in the paper). Authors should strive to give a balanced and fair assessment listing both strengths and weakness of all work considered.

Example Paper Types

• New transformation technique: The most common type of PEPM paper will report on a new analysis/transformation technique. Such papers should include a detailed description of the technique, some sort of formal or informal argument about why the technique is correct, illustrate the technique on interesting examples, and evaluate the effectiveness of the technique.
• Case studies: This type of paper reports on a large-scale application of an existing technique. Such papers should clearly state the insights and the lessons of the study. Was the technique effective? Was it easy to apply? What pitfalls were overcome and what difficulties remain? One must clearly distinguish anecdotal performance claims from rigorously established ones.
• Description of new/interesting domain and potential for effective use of PEPM techniques: This type of paper describes a particular application domain (e.g., middleware, avionics systems, active networks) or a particular development paradigm (e.g., model-driven development) detailing how techniques within the scope of PEPM might be applied effectively in that domain. Special attention should be given to enumerating the problems/challenges of the domain, characteristics that suggest that it might benefit from PEPM techniques, initial attempts at applying those techniques, and questions/challenges to be addressed by future work. Such papers should aim to educate the PEPM community about interesting/fundamental issues of the domain, and give pointers to where they may learn more about the issues.

Short papers

We do encourage submissions of "work in progress" in cases where the submission raises issues that will generate interesting discussions at the meeting, brings new knowledge of a particular application domain or technique to the community, or lays out challenging open problems of high relevance to software engineering practice. Depending on the quality and number of such submissions, we may collect work-in-progress papers into a single session with slightly shorter time slots for each presentation and a longer discussion time at the end of the session.