Latest revision as of 20:37, 30 January 2018

This document is meant to give a simple check list for paper writing (journal articles) in cognitive science and neuroscience, originally put together for the Kording lab. It includes a range of insights I adapted from the (highly recommendable) and more general list by Dan Simons. Also added some ideas from Tony Zador. Comments are welcome. Keep in mind that these rules (under rare circumstances) may be broken.

Global:

1. Make the central ideas be sticky. As the book "Made to stick" by Chip Heath and Dan Heath argues, there are certain factors that make an idea sticky: Simple, Unexpected, Concrete, Credible, Emotional, Story. Try and maximize this stickiness for the central aspect of the paper.
2. Any journal paper should be either about one idea or about one method. Papers with more than one topic usually dilute the main message and are hard to read. If other points need to be made (e.g. understanding DNA incidentally allows understanding how it is replicating) then it should be made in one peripheral paragraph.
3. A paper should have a well defined target audience, e.g. scientists (e.g. send it to nature), neuroscientists (e.g. neuron), neuroscientists who work on movement (e.g. experimental brain research), neuroscientists who work on feedback mechanisms in the context of movement (e.g. Journal of Motor Behavior)
4. Do not claim “first.” Many inexperienced authors strongly emphasize that they are a first to do something. This is strategically bad. (a) I find that the majority of claimed “firsts” are actually wrong. Stating “first” just makes the referee search hard for prior art and “first” often requires very clear definitions to be true. (b) In some sense claiming “first” implies that it is not blatantly obvious that you are first because your lab’s work is so cutting edge :)
5. One of the important issues is to realize that the issues that you spent a lot of time on are often quite peripheral to the message of the paper. The reader will have a much harder time understanding the stuff than you think. After all you spent many months of your life thinking about the issue. For each project you will in some sense be the best specialist on this planet. Even famous people have large trouble understanding other scientist’s studies.
6. Parallel ideas should go with parallel language. For example, if your data has two interpretations, you might want to write a paragraph about each. These paragraphs should have a structure that is as similar as possible. Similarly, if you have several sentences describing different possibilities, they should have the same grammar. And if your thesis has chapters on related topics, structure the chapters in the same way.

The triple C way of thinking:

1. The paper, along with each section and each paragraph, should start with something about context; continue with stuff we already know; present a logical progression of argument; and then summarize with something like a conclusion. This way of writing is called triple-C “Context, Content, Conclusion”.
2. Every paragraph must be about one topic and one topic only. The first sentence should introduce the topic. The last sentence should summarize the answer about the topic. The sentences in between should define a clear logical link between the Context and the Conclusion.
3. Triple C is particularly useful as psychological studies show that beginnings and ends of lists are particularly well remembered. Your reader will remember the central idea as opposed to peripheral information.
4. For further info see: 3C

No duplication, Zig-zag or excessive links

1. Minimize zig-zag; find a logical way of ordering your ideas.
2. An idea should generally be presented at one (and only one) place in the paper. If an idea is particularly important, it might have to be revisited to keep it in the readers mind.
3. The one big exception is the central idea of the paper which is exposed in the intro, motivates the methods, is supported in the results, and the implications of which and problems with get discussed in the discussion.
4. Minimize the use of links that readers are not actually going to follow: "see methods", "as discussed above", "see page 3" etc. The reader will not interrupt their flow and it just wastes space.

Paragraphs:

1. Each paragraph is about only one idea. The first sentence exposes the idea, the last sentence concludes it and the rest contains the data/ideas/citations to clarify that idea.
2. Each sentence in the paragraph contributes to that idea.
3. Any paragraph should be at least 4 lines long (shorter feels like it is some stray sentences). In rare exceptions 3 liners can be necessary (e.g. to describe what a chapter is about). A paragraph should generally not be longer than 12 lines as it then is hard to follow.
4. Every sentence is necessary to support the idea of the paragraph.

Sentences:

1. Do not use brackets that contain more than 6 words.
2. Use parallel structure across sentences whenever possible.
3. Use active voice whenever possible.
4. Do not try to make the language sound scientific. Say things simply.
5. Use short sentences. No sentence should be longer than 3 lines.
6. Explain why something is important or interesting instead of saying that it is.

Words:

1. Do not try to use scientific words, use simple words.
2. Do not use “rather”, “very”, “little”, or “pretty” unless you are making a joke, grading an exam, mean the opposite of large, or are talking about beauty.
3. Do not use acronyms if you can avoid it. If you need to use them, use only as few as possible.
4. Avoid jargon. This will make the paper more readable despite being less precise. Jargony words can essentially cause readers to skip entire sentences or paragraphs, and this can cause them to miss important points. So it's best to use as simple of language as possible throughout the paper, and to ease into precision both within a paragraph and within the paper. Introductory sentences/paragraphs should be easy to digest and paragraphs/paper should only then dive into more detail. You may want to ease out of precision back into readability at the end of paragraphs/papers as well.

Workflow:

1. Spend a lot of time thinking about the idea and fine tuning it before starting a paper.
2. Write the abstract before starting any data collection. We first need to be sure the idea is worth writing a paper about and, given good data, makes an important and clean message.
3. Start with a complete outline before writing the text.
4. Print the paper and read through it before submitting it.
5. Have others read the paper before submission. Bribe them if necessary.

When in doubt, unpack ideas

1. Your audience is from a different field. They will thus not have the intuition behind an idea. Whenever something is potentially unclear it should be unpacked.
2. You are making a claim that wouldn't be universally accepted. Unpacking here might consist of a few sentences that provide some justification for that claim.
3. You introduce a new-ish idea in the paper. Unpack early (i.e., in the introduction).
4. In general, you want to unpack ideas if your mom would not understand it the way you wrote it.

The anatomy of the paper:

Today's journal articles in cognitive science, molecular biology, neuroscience and many other disciplines all have almost identical structure. This makes it far easier for experienced readers (those reviewing papers) to read the papers, fact check them, check if they are complete and check for novelty. Sticking to the structure makes publication success dramatically more likely.

Abstract:

1. Written for a very general audience. No jargon. Write for your grandmother. Must take the reader by the hand.
2. First sentence is about the big field.
3. Introduce big open question.
4. Explain basic experiment/analysis idea
5. Explain methods to be used.
6. Explain results
7. Last sentence is about how this changes progress in the field

Intro:

1. Start with the most general aspect, e.g. one of the central questions in neuroscience is how the interactions between neurons allows animals to learn.
2. Every paragraph starts with a sentence describing what generally follows. Usually the first sentence is an implied question but without a question mark. E.g. “A central objective in neuroscience is to understand learning”.
3. Every paragraph ends with a sentence that says “therefore the study you are about to read is important.” Stuff that does not scream – this is why the central idea of the paper is important! – or does not flow well is better off in the discussion.
4. However, first and last sentence must be at the same level as the paragraph. So if the paragraph is about neuroscience and learning the last sentence must be about the wide field. If the paragraph is about the lack of tools to probe how priors are acquired then it needs to end with a statement that such tools promise to improve the interpretation of previous experiments that measure priors.
5. Progress towards more specific topics, e.g. motor adaptation as a way to study learning. Go progressively more specific
6. The last paragraph of the intro is special. It starts with a “Here we”, explains the methods to be used and the results obtained. It does not go into any details and does not discuss the results.
7. Intro should never be shorter than about a page. Aim for 1.5 pages. But neither should it be longer than say a quarter of the paper.
8. Do not summarize the literature you know. Summarize the literature that sets the stage for your paper. Bring across why your research matters.

Methods:

1. Everything should be reproducible based on the methods
2. Every paragraph should still give an intro on why this is important or necessary. For example it might be “We need a method to quantify if people rely on feedback and to do so we use …”
3. The methods section usually does not have a figure of its own, sometimes the section is at the end of the paper. It is fine if the methods refers to a figure that comes later in the text. This main methods figure (usually fig 1) is usually then really cited first in the first paragraph of the results.
4. In our lab I want to share data and models. Keep that in mind when working on the paper. All data analysis scripts and model scripts must be of high enough quality to be made available to others.
5. Methods should be sufficient to replicate but should not contain too much unnecessary information. It might be important to say we use “Matlab” but no need to say “custom made scripts”, “Pentium 4 running at 2.3Ghz”, etc. Only stuff that would be relevant for replicating.
6. When human subjects are used it needs to say that informed consent was obtained and the study was performed according to institutional and national standards for the protection of human subjects.

Results:

1. First paragraph summarizes the central question and the methods to be used.
2. The paragraphs in the results section should start with essentially a characterization of the raw data and progressively evolve to a quantification for single subjects and then the group of the central hypothesis. If necessary, the end paragraph may even analyze some implications of the central idea.
3. When referring to figures do not use “Figure xxx plots x vs y and shows z” as this doubles the figure and figure legends and is wordy and hard to read. Rather refer to figures as in “Using our experimental paradigm we find that uncertainty increases learning speed (Fig 3b).
4. Every paragraph starts with a question, again usually not in question form. It ends with a statement that has larger meaning. E.g. “We wanted to test if uncertainty affects learning speed” and “Uncertainty thus clearly affects learning speed”
5. Roughly the results section should have 1 paragraph per panel – unless a range of panels have a common message in which case they should all go into one paragraph. However, if the resulting paragraph would be more than 12 lines it is better to not merge the discussion of the graphs.

Discussion:

1. First paragraph summarizes results at a level slightly more abstract than the last sentences in results.
2. Then there are a couple of paragraphs that deal with a range of things. (a) Fundamental problems of the study. Discusses the problems and why the study did not do better at dealing with them. Highlight how future research could deal with those problems. Be aware that reviewers might ask for those things to actually be done in the study. In particular for high level journals. (b) Discuss relevant other literature. Highlight how it asks the same questions but from a different angle. Or how it extends previous studies. (c) Discuss how the given study requires a rethinking of past research. (d) Discuss how this opens new avenues in other areas for the scientific study of the problem at hand. In all cases – emphasize how the given study does make definable progress.
3. Be dismissive neither of the to-be-published story nor the stuff of others.
4. Discussion should generally not be less than 2/3 of a page. It should not generally be longer than ¼ of the paper.

References:

1. Make sure that relevant fields are cited and usually each field deserves an intro or a discussion paragraph but usually not both. If you completely miss on one field people will assume bad scholarship.
2. It is usually not wrong if the most likely referees are cited.
3. Make sure to cite people and studies who may disagree with the results of the study at hand. Explain why there is a disagreement. Do not declare the disagreement finally solved by the study.
4. A typical study should have more than 20 but less than 50 citations. Unless it is very short (nature allows 30) or very long, say a long review article may have 300 citations.

Figures and Captions:

1. Each figure deserves to look good. So make figures in Matlab/Poser/whatever. Then load them into illustrator to make stuff look good. Spend a few hours on each figure. It is well worth it.
2. First figure should explain the experiment/methods used.
3. Use insets and graphics to explain experimental ideas whenever possible. A figure is almost always preferable to text.
4. Figures are usually viewed in a small form. So use large fonts.
5. It is often a good idea if the caption of the figure allows understanding the conclusion from the figure.
6. Ideally, the figures allow understanding the entire paper without reading.
7. Even although journals generally say that the figures should be submitted at the end of the text, most reviewers agree that putting them into the pdf of the main text makes reading easier. A good number of editors agree informally.
8. Use the same font across all figures and at most 3 sizes.
9. Typical papers will have at least 3 or 4 figures and no more than 8. A figure should not have more than 8 panels and usually at least 2.

Responding to reviews:

1. Workflow: Start with reading all the reviews in detail. Then write a complete answer to all of them. Only then do additional experiments/analysis.
2. Understand that reviewers usually actually are trying to do their best (but they are often under heavy time constraints).
3. In the letter to the editor emphasize and detail how the reviewers comments have allowed you to improve the paper. This is particularly important if they have not.
4. In the response to each reviewer start with about 4 lines of text that states that you want to thank the reviewers for their constructive criticism/ deep insights. Give a very short high level intro to the big revisions
5. Answer each of the reviewers points. If the reviewer produces long text feel free to summarize the text. Often verbose reviews are quite repetitive and summarizing makes them more manageable
6. You really want to demonstrate that you put a lot of work into the revision. Show additional figures. Detail how you did additional analysis.
7. Every comment deserves a response. If the comment is stupid it still should be taken constructively. A stupid comment usually just means that the reviewer couldn’t understand the paper because it was too hard to read. If the comment seems stupid do rework the portions of the paper the reviewer criticized. Accept culpability in the response for writing in a confusing way, and state that this was corrected in the new revision and that you are sure it is much better for the readers now.

This ends the paper writing check-list. I hope it will be useful for the papers you all write. By the time a paper ends up on my desk none of the rules above should be violated. Apart from those that really need to be violated :)