Fuzzy COI

In Scientopia, I use a reader's question about whether to serve as guest-editor of their advisor's and other close colleagues' manuscripts (short answer: no) to discuss more ambiguous examples of possible conflicts of interest (COI).

Bio Hazards

Some grad students and postdocs who write to me have advisors who abuse the advisor-advisee relationship by making unreasonable demands on their advisees regarding how they spend their work time, how their time and contributions are credited in publications, and (in the case of grad students) how much time they spend as students before being awarded a degree. In some of the worst cases, the advisors make specific or veiled threats that their future letters of recommendation for the student/postdoc will only be positive if the student/postdoc does what the advisor (unreasonably) demands. The students/postdocs are trapped, fearing for their future careers. (This fear of letters of recommendation also occurs in other situations.)

Most, but certainly not all, of these emails are from students and postdocs in the biological sciences, and specifically biomedical/biochemical fields. As I've discussed before, that could be because there are more people in these fields, or it could be because these fields have a greater propensity for misbehavior, based on the size and structure of the research labs. I don't know (but there are many interesting comments on the post linked to previous sentence).

Some of the scenarios described in e-mails sent to me by anxious students and postdocs seem very clear-cut examples of unethical behavior by advisors. I hope that in at least some of these departments and institutes, there is a mechanism by which students and postdocs can get help and these advisors be held accountable for their repeated 'irresponsible conduct in research'. It doesn't seem like there is, or, at least, it doesn't seem like students know about the possibilities and/or feel that there are any reasonable options.

And that leads me to my question of the day: What are the mechanisms by which advisor-advisee problems such as these can (theoretically) be resolved? 

I am particularly ignorant about the biosciences, but there is likely quite a lot of variability in this answer as a function of institution type and size, across all academic disciplines. My impression, however, is that large biomedical/biochemistry/bio-etc. labs operate as semi-autonomous units in which the PI sets the rules, even if some of these rules would be widely recognized as abusive. (tangent: Don't these PIs have to undergo Ethics Training and attend Responsible Conduct in Research workshops? Don't they see themselves in some of the classic case studies discussed in these workshops?). Advisor-advisee problems can be difficult to resolve even in departments in which there is some degree of oversight, but what recourse do students have in PI-ruled kingdoms?

Even in cases in which there is a mechanism by which students can get help from faculty and administrators, the most difficult issues remain: a student who has a lot of time and effort invested in a research project doesn't want to have lose ground and experience even more disruptions (i.e., sometimes it is easier just to put up with an awful advisor and get out as soon as possible); and
a student with an abusive advisor may be reluctant to complain, fearing the consequences for future career opportunities (the letter of reference anxiety, common to all types of harassment situations).

I will give one specific example of a conflict and resolution situation with which I am familiar, just to show that it can work out in some cases. At University Z, an advisor didn't think a grad student had done enough work to attain the degree, even though the student had been in the grad program for longer than peer students and had (in the opinion of some) accomplished at least as much as others who were awarded the same graduate degree from that department. The faculty had some sympathy for the advisor, who was justified in being frustrated by the not-great work done by this student, but the student had a job offer, and the offer depended on the student's having the graduate degree. The advisor was not willing to budge; the student needed to do more work or no degree, even though the advisor had no grant to support the student's further research. The rest of the student's committee and the departmental graduate program advisor examined the available information and disagreed with the advisor: they felt that the student had done enough research of sufficient quality to be awarded the degree. The advisor refused to sign the relevant forms, so the graduate program advisor and committee members took care of the paperwork, the student got the degree, and is now happily employed.

This worked out in part because the student no longer needed the advisor's letter of reference; in the future, if a letter is needed, the graduate committee members can help with this, bypassing the angry advisor.


That incident had a happy ending; I suspect that most do not, but it would be useful to have more information, in part so that we are all better informed when students ask for advice.

Therefore, I am interested in hearing about examples from different departments, institutions, and academic disciplines:

• Is there a good way to resolve serious problems in your department/university/etc.? 
• How much is your department (etc.) willing to get involved in advisor-advisee issues, when problems are detected? and
• Even if there is some process on the books for resolving serious problems involving advisors and students, do students ever find it reasonable and useful to pursue these options, or is the conflict resolution process seen as something that sounds good in theory, but has too many pitfalls (or is just a bunch of empty words in a document somewhere, to make administrators feel better)?

Citing Creeps

Today in Scientopia: On creeps and their effect on the work of others

Great Peace

As many of my fellow bloggers know, we are constantly sent spam "comments" from people offering their services as dissertation writers, researchers, and editors. As my fellow bloggers also know, 99.9% of these enticing offers look something like this:

Great peace of fact about to done by one of my recent PhD search and explaining,information provided is also brilliant. [link to] Dissertation Introduction

or

well your Po$t is good and i really like it :). . .awesome WORK . . .KEEP SHARING. .;)[link to] Dissertation Editing Services

I can't say that I have never seen a thesis written like that (alas), but I can't decide if it makes me feel better or worse to think that someone might spend money for writing assistance that looks like that.

Maybe the guys who try to get their spam ads posted as blog comments aren't actually doing the dissertation writing or editing. Let's consider whether someone who writes well in the relevant language(s) could write/edit a scientific dissertation.

There was a recent article in The Chronicle of Higher Education by someone who clearly writes well and has considerable skills as a researcher. Ethical issues aside, that person no doubt provides value for the money, although none of the examples listed in the article involved writing about the results of scientific research performed by the person hiring the ghost-writer. It mostly seemed like the 'research' involved interpreting the results of literature searches for undergraduate and MS students; i.e., the kind of thing you can write by reading Wikipedia and a few other sources and pulling it all together if you put some thought and time into it.

This requires skills (thinking and writing), but can someone with no significant background in the sciences write a convincing document (dissertation, manuscript, proposal) involving original scientific research?

Some of my students have sought writing help from various on-campus resources or friends who are not scientists. If given a document that already contains the data, equations, jargon, citations, and so on but that needs help with the technical aspects of writing, certainly a technical writing expert can help improve the document if they are generally aware of the conventions of science writing. And such writing support can help a lot with fixing basic problems encountered by those who don't have a lot of experience writing in a particular language. I am all for technical writing assistance where needed.

But can such a person write a good Science dissertation introduction for someone else? Or a discussion? What about the abstract? I am skeptical that a non-expert could write a convincing intro or discussion, but maybe they could write a good abstract.

Or am I delusional about this?

Potted Plants

In the Sunday New York Times Magazine, "The Ethicist" fielded a query from a grad student who wondered about the practice of offering undergraduates extra credit to attend a conference hosted by their department. At the conference, "high-level" talks were presented; talks that most of the undergrads would be unlikely to understand.

The grad student thought that the undergrads were being exploited, as the main purpose of offering them extra credit was to increase the number of people in the audience, thereby perhaps convincing the university to increase funding for the conference in future years.

The grad student disapproved of this practice, and The Ethicist agreed, stating "You might as well fill the audience with potted plants."

Like many of you, I have given many talks attended by undergraduate students (and others) who did not have the necessary background to understand everything I presented. In some cases, these students were there by choice, and in some cases they were there for extra credit. Most typically, students lured by credit (extra or not) are undergrads, but I have also given lectures to audiences that consisted in part high school students who were required by their teacher to attend a Science Talk given by a Real Scientist.

Were these students potted plants? Perhaps some were, but many were not. The fact that at least some students are paying attention and trying to understand as much as they can is particularly obvious in settings in which questions are encouraged, even from non-experts. For students not comfortable asking a public question, there may also be opportunities for questions and other interaction after the talk. These can be very interesting discussions.

And even for those students who are unable to understand much of the content of a talk, there may be some value in attending a talk or a conference. In these settings, students get to see other academics in action, get a sense for the content and style of presentations, and observe how researchers interact with each other.

I encourage undergraduates who do research with me to attend conferences, including national conferences in our field. These students attend talks, many of which they don't understand, but they report being intrigued by the whole conference/cultural experience. I have also found that undergrads are quite adventurous about which talks they attend, sampling talks on topics that represent a wide range of sub-fields, to see what's going on, what's hot, who is interesting, who is not. Some of it is boring and much of it is puzzling, but it's also kind of fascinating.

I therefore disagree with the conclusion that undergrads who have insufficient background to understand high-level academic talks are passive potted plants, as many probably are getting something out of attending a talk or conference in their general field of interest.

Even so, to avoid the possibility of potted plant syndrome among extra credit-seeking undergrads being exploited by funding-hungry conference organizers, perhaps the experience could be enhanced somehow. Perhaps the students could be prepared in advance for some of the talk topics. Perhaps the students could meet some of the speakers after the talks for additional questions in a less formal, more undergrad-friendly environment. Perhaps speakers could be encouraged to spend the first couple minutes of each talk giving background information at a more basic level than they otherwise might. Perhaps everyone would find that more interesting.

I think the attending of talks and conferences by interested undergrads should be encouraged. I don't tend to give extra credit in my classes other than a 1-2 point fun/strange question at the end of exams, but, as long as extra credit isn't a significant portion of the grade, I don't have any problem with there being some sort of credit given for an academic experience that might well be enriching.

Reviewer Know Thyself

A manuscript that was submitted earlier this summer was recently returned from review, and lucky for us there were many positive comments about the work. There were also some very negative comments from one particular reviewer.

This reviewer, who indicated his identity in the review and recommended that we cite more ofpapers his , didn't really care about the cosmic implications of our work. He focused intensely on one particular method. He wrote in detail about the assumptions of this method, giving us a little tutorial about the method we used. Apparently his research is an excellent example of the appropriate uses of this method. Apparently, our research is deeply flawed.

We disagreed with every one of his comments, and wrote a detailed response letter to the editor, who accepted the paper.

By complete coincidence, as part of an entirely different project, some colleagues and I found ourselves revisiting the research of this reviewer, including the specific subject of the paper he held up as an awesome example of the appropriate use of the method in question. We had not intended to check into his research so closely, and the timing of his review relative to our delving into his research was unintentional.

Nevertheless, the term 'house of cards' kept coming to mind as we delved. Many of the issues that the reviewer raised as problems with our application of the method were present at similar or more serious levels in his own work.

I think that his work still has a lot of merit. The study he did was important and the resulting publication is interesting. Nevertheless, it was extremely hypocritical of him to take our manuscript apart as he did, and hold us to a higher (impossible) level of purity -- a level he himself did not even approach in his own work.

I don't know this reviewer well, so I have no insight into the psychological aspects of his review. Does he really think that his paper lacks the flaws he sees in ours or does he know this but prefers to pretend otherwise?

Is it ever reasonable for a reviewer to hold others to a higher standard?

In fact, in some cases it is. For example, early studies to explore a new method or a question may be a bit rough; later work should improve on this if possible.

That wasn't the case with the review in question. In this review, the researcher was being pedantic and hypocritical. We were fortunate to have an editor who weighed the reviews, positive and negative, and decided that the negative comments were not reasonable. Hooray for thoughtful editors.

Retract?

An FSP reader has a conundrum.

Imagine that you find yourself in this situation (anecdote condensed and edited from the original e-mail): A friend and co-worker has lost his/her job. You think this person has been treated unfairly, but you don't know the official facts of the case. Rumor has it that this person had done unethical things, but you have worked closely with this person and had found him/her to be very professional and reliable. On the basis of your own very positive professional interactions with this person, you write positive letters of reference for faculty jobs.

You later receive credible information indicating that this person was in fact lying about a great many things to a great many people, and that there was substantial unethical conduct. You can no longer stand by the statements in the letters of reference that you sent on this person's behalf.

This person claims to have found a position that will begin in the fall, but you don't know if this is true and there is no one at that school whom you can ask informally. A web search doesn't turn up anything, but it probably wouldn't for a recent hire anyway.

Do you have an ethical obligation to contact the schools to which you sent the positive recommendations and inform them that you can no longer stand by the statements to which you signed your name? Would search committees and department chairs want to know this?

Aside from the ethical considerations, what might be the consequences of these now-discredited letters for your professional reputation, especially if you are an early career faculty member trying to get established? Are the consequences worse if you retract your statement or if you do nothing?

What would you do?

First, I want to reassure my correspondent that he/she should not regret writing the positive letters. The letters were written in good faith, based on personal experiences working with the colleague in question. The letter no doubt mentioned these positive interactions, perhaps providing specific examples. If there were no indications of unethical behavior in these interactions, the letter was a fair statement of what the letter writer knew to be true.

If the letter stayed close to the specific interactions that the letter writer had with this colleague, perhaps a retraction is not necessary. If, however, the letter writer made some broader statements that really should be retracted, perhaps a brief letter should be sent to the relevant search committee chairs and/or department chairs.

The letter need not elaborate on the situation, but could just say something like "Owing to facts that have come to my attention since I wrote the letter on behalf of Colleague X, I can no longer stand by the positive letter that I wrote on his/her behalf. Please retract my letter, if this is possible and relevant to your search." You can decide whether you are comfortable mentioning that you are willing to provide further explanation if necessary, and then list your phone number.

If Colleague X has a new job, there must have been more than one positive letter.

Another possibility is to discuss this with a trusted senior colleague -- e.g., a mentor or the department chair. Perhaps they can intercede in the situation, especially if there is a senior person with more direct knowledge of the unethical doings of Colleague X who would be willing to help figure out the best course of action.

But perhaps various readers will have various other suggestions..

To What End?

It is time for the Evil Reviewer anecdote to which I alluded in a post last week.

Colleagues and I submitted a manuscript about our Transformative Science to Awesome Journal. We were pleased to make it to the review process in a journal that summarily rejects most submissions.

We waited. And waited.

The reviews came back. The identities of both reviewers were very clearly stated in the reviews. Famous Scientist 1 loved the paper. Famous Scientist 2 hated the paper. We were allowed to revise the paper.

We revised the paper, taking into account the more rational of Famous Scientist 2's comments and providing a rebuttal to those we considered unreasonable.

The paper was re-reviewed by the same reviewers. This took a long, long time. Nearly a year had gone by before we received the second round of reviews back.

Famous Scientist 1 loved the paper even more. Famous Scientist 2 hated the paper even more. The paper was rejected.

The primary author (not me) protested. In fact, he was enraged because FS2's comments were rather inflammatory and his most serious criticisms were demonstrably untrue. One comment was that the questions we addressed were not of interest, and the methods we used were flawed.

We argued against the criticisms and other comments that could be demonstrated to be untrue, using citations of recent papers about the ongoing, unresolved debate on the topic of our research. The editor agreed to send the manuscript to a third reviewer who would have access to the previous reviews and correspondence.

Reviewer 3 (whose identity is not known to us) could not believe that FS2 would deliberately shoot down the paper unless he had solid, scientific reasons to do so. The editor's decision to reject the paper (again) hinged in large part on this belief that FS2 was an "honorable" man.

In the meantime, FS2 was busy. He reproduced our results, addressing the same questions (which he had stated were of no interest to anyone) and using methods he had criticized as flawed in our work. He submitted his own manuscript on this same topic to another journal. This fact came to our attention by chance.

What to do?

My colleague, now even more enraged, was able to document the existence of this "new" manuscript, complete with date of submission, showing that at the very same time FS2 was taking an unusually long time with the review of our manuscript, he initiated research on the same topic, submitting his own manuscript soon after his second, savage review of our manuscript. My colleague wrote a long, detailed letter to the editor.

The issue remains unresolved, but it is unlikely to have a happy ending for anyone.

Fortunately, in the case of our beleaguered manuscript, none of us authors are at a career stage that hinges on having a high profile publication. We think the science in the manuscript deserves publication, and we think we deserve credit for the ideas and applications in the paper, but it doesn't really matter where the paper gets published.

It is difficult for me to understand the level of ego? enmity? selfishness? depravity? that would drive someone to say that our research was worthless, and then immediately turn around and work on that same topic using the same methods (and !surprise! get the same result). We have all lost respect for Famous Scientist 2.

This episode made me wonder:

Was it really worth it to him to do this just so that he could be "first" to publish these results?

What am I supposed to think about the other papers by this Famous Scientist? I don't doubt that much of his existing work represents his own ideas, although now I wonder if he has ever done something like this before. I have met him and I know him to be a very smart and creative scientist. He doesn't need to steal the work of others.

Do they have "responsible conduct in research" workshops at his institution? Does he attend them and feel gnawed by guilt at his unethical conduct or does he glaze over in boredom at the case studies of postdocs in biomedical labs, thinking that none of this relates to him?

How open should I be with other colleagues, students etc. about this situation? Assume that there is no reasonable possibility that we misinterpreted FS2's actions and that there is solid evidence for the scenario outlined here. Being silent about the situation might allow him to continue in his evil ways, potentially doing real harm, but openly accusing someone of dishonesty (however much documentation I may have in my files) sets up a situation of "Which person do you believe?". The answer might well be "Famous Scientist 2" (as we saw in the review process).

Is FS2 pleased with himself that he was apparently successful at getting our paper rejected, or does he feel any discomfort at all about his tactics? Or maybe it is all just a game to him, and he feels satisfaction that he has apparently "won" this round?

I don't know. In my career, I have seldom experienced anything so blatantly inappropriate associated with a manuscript review, so I prefer to think that this is a relatively rare situation. I regret that I can never again contemplate this Famous Scientist's work without a hostile suspicious feeling.

I hope that our paper will eventually be published and appreciated for its interesting science, primarily the work of my colleague, who deserves full credit for his creativity, hard work, and persistence.

Playing the Game (2)

Citation index considerations aside, have you made any publication decisions specifically because you wanted to increase your number of publications, on the cynical-but-perhaps-justified assumption that more papers = better?

For example:

1. Have you taken what probably should have been one paper and split it up into two or more shorter (but nevertheless distinct) papers for the sole purpose of increasing the total number of papers on your CV?

Figuring out how many papers should be written about the results of a particular research effort is a bit of an art, and in some cases more papers is the best decision. Sometimes you know in advance whether to write results up as one or more paper(s), and sometimes you don't know until you start writing.

My question, therefore, refers to cases in which the main motivation for splitting up a project into n>1 papers is to crank up the publication numbers.

2. Have you published what probably should have been one paper but that instead were two or more related papers that were not as distinctly separated in content as in the first scenario? (i.e., shingling)

This is also a fuzzy concept because sometimes you want to publish one short and zippy paper and then another, longer one in a more specialized journal. Is that shingling? Or what if you decide to publish some different-but-related papers because each involves different groups of co-authors and it makes logistical sense to keep the publications separate? Or maybe you want to publish a review paper that summarizes information in some of your other papers. There are many legitimate ways in which similar papers are published by the same author.

But then there are other cases. I recall one time when I brought a manuscript to review on an airplane. This was back in the Days of Paper, so I had a hard copy of the manuscript, and had turned to a page with a figure on it. I was traveling with some colleagues to a conference, and a colleague in a different-but-related field was sitting next to me. He was also reviewing a manuscript, and turned the document to a page with a figure on it. It was the exact same figure. Yes, I know that manuscripts in review are confidential, but there we were, sitting next to each other with two different manuscripts submitted to two different journals at the same time, and there was at least one thing in both manuscripts that was identical. So we started comparing. The two manuscripts had the same authors, the same figures using the same data for the same topic, and only a slightly different "spin" put on different aspects of the data. Those two manuscripts clearly represented shingling of a sort that was probably not OK. We informed the editors of the journals.

3. Have you ever submitted a paper before the project had advanced as far as it probably should have before writing up part of it as a manuscript; i.e., a premature manuscript that was probably publishable but that you knew would be much better if you worked on the research more? (but you didn't feel you could afford to wait longer because you needed publications on your CV sooner rather than later)

As with the other cases, this is not an obviously bad thing to do either. Maybe you would have waited longer to publish if you already had tenure, but there are also good aspects of publishing a preliminary paper to communicate initial results rather than waiting, perhaps years, to publish one big definitive paper. You'd want to be as confident as possible that the preliminary paper was sound, but if you feel you have something to say that is of interest, I think it can be very useful to publish early and often.

I know that that is "playing the game" and perhaps contributing to the mass proliferation of academic articles so that the flow of information is overwhelming and the very act of publishing a scholarly work is devalued etc. etc., but I think that if you have something interesting to say, it's a good thing if you write it up and get it out there.

Ideally, in the course of your career, you will publish some short papers and some longer, more detailed papers and some review papers and some papers in Awesome Journal X and some other papers in Specialized Journals Y and Z, and it will all even out. It's in the early stage of an academic career when every decision about what/where/how much to publish can seem so critical.

From what I've seen in the physical sciences, the "best" route to take at all stages of an academic career is to try for a balance between publishing a reasonable number (according to the norms of your specific field) of very good but perhaps not awesome papers in respected journals, and then some (but likely fewer) rather awesome paper(s). This is preferable to having lots of narrowly focused papers or only/mostly having "big idea" papers. Together, however, these different types of publications demonstrate the breadth and depth of your research.

When deciding how to divide up a big project into papers, I try to optimize publication quality, speed, and impact, as well as consider what is most fair to the most number of co-authors, particularly students, postdocs, and/or tenure-track colleagues. This not always a straightforward decision, of course, and sometimes I wonder if I would do just as well by consulting a Magic 8-Ball for advice.

Or maybe I should go back for yet more training in Responsible Research Conduct. Surely somewhere in all those case studies and PowerPoint presentations, there is a nifty formula for sorting all of this out easily. {note use of delusional font}

Playing the Game (1)

This spring, I have to refresh my ethics training, a ritual that is annoying mostly because the university makes it so extremely difficult for faculty and postdocs not involved in the biomedical sciences to find relevant ethics training opportunities. I am really not that interested in spending 2 days learning about research with human subjects, as my research involves no human subjects (other than grad students..).

And now all our graduate and undergraduate students who receive a salary or stipend from NSF must be trained in ethics.

Ethics training is important, but it is boring: don't plagiarize, don't fabricate data, don't falsify data. And the case studies are bizarre.

So let's forget about whether the postdoc should hide the data outliers that may or may not be due to a power fluctuation during extremely expensive analyses at a national lab even though the grad student involved in the research thinks that they should at least inform their PI, and consider some more interesting situations. For example, following on yesterday's citation theme:

Has your awareness of the importance of citation data affected your decisions about publishing?

That is, would you have made a different decision about anything related to publications (authorship, type/number/format of publications etc.) if your citation metrics were not being constantly tabulated for all the world to see?

When answering the first-order question, consider only whether you have ever considered the impact of a publication decision on your citation indices. Such considerations are not a priori unethical; it is possible that such considerations will result in your making a completely ethical and smart decision that helps your career. An example of that might be to send a manuscript to a journal that is indexed instead of to an edited volume being published as a not-indexed book. You do not change anything about the authorship or content of the paper; just the publication venue.

But now consider whether you have ever made a citation-fueled publication decision that might not have been entirely ethical but that could possibly be justified because "That's how the game is played." (others are doing it; if you don't play the game, you lose; etc.).

I don't think I've made any unethical decisions about research publications, but I definitely had citations in mind a couple of years ago when I decided to revise a much-cited technical document that was originally authored by someone else (now long retired) and that was in great need of an update. In that case, though, I wasn't thinking so much of my own citation index, but was instead intrigued by the idea of transferring citations from Journal 1 to Journal 2, in part because I am involved in the editing of Journal 2 and therefore have an interest in promoting the success of Journal 2. I hasten to add that I am paid nothing for my editorial work for Journal 2, so financial considerations were not involved. There were in fact valid scientific reasons for publishing the update in Journal 2, in addition to my personal affection for Journal 2. In the end, though, I published the update in Journal 1, but I admit that my original intentions were rather craven and driven in part by thoughts of citations.

That's the worst example I can think of for myself, but perhaps that is because I am a tenured mid-career professor. If I were on the tenure-track today, I am sure it would be difficult not to consider citation index issues when making publication decisions, although if it makes any of my early-career readers feel any better, I happen to know that at least some promotion & tenure committees at research universities are specifically instructed not to consider the h-index or other citation statistics when reviewing files for tenure and promotion.

to be continued..

Scientific Overlap

It recently came to my attention that a certain scientist had submitted identical proposals to two different funding sources at the same time. Submitting "overlapping" proposals to the same funding agency is prohibited by some (but not all?) funding agencies.

Scattering the same proposal among different programs in the hopes that one (or more) of them will fund the proposal might seem like an efficient strategy for the PI who invested a lot of time in a proposal, but this practice is prohibited for good reasons. Given the time and effort involved by funding agency employees, panel members, and reviewers to deal with the large number of submitted proposals, a prohibition on overlap makes sense

NIH outlines in detail the consequences of the simultaneous submission of proposals with scientific overlap. I did a rather quick search of the NSF website to try to find a similar document, but didn't find anything. It seems like there must be a policy, though, so maybe I just missed it.

Rejecting proposals for reasons of overlap is straightforward if the proposals are identical or even "essentially identical" (in the words of the NIH document). It might become more difficult to judge overlap when the definition is extended to proposals containing "similar" research, another term used in the NIH document, but I suppose experienced program directors know it (overlap) when they see it.

In the case to which I referred in the first sentence of this post, there was no administrative mechanism by which the "overlap" could have been detected if a reviewer had not been become aware of both proposals. I suppose there is a high probability of a reviewer's discovering overlap in cases in which the research is highly specialized and both funding programs use peer review. The same reviewer(s) are likely to be sent both proposals.

In the first sentence of this post, I used the vague term "funding source" deliberately so that I could consider the ethics and consequences of the following situations:

- Identical proposals are sent to different programs of one funding agency that specifically prohibits submission of overlapping proposals. This is clearly wrong and both proposals would be rejected

- Identical proposals are sent to different programs of a funding agency that does not have specific rules about overlap. Reviewers might balk at this (if the overlap is detected) because it is annoying and seems unethical, but if there are no rules against it, what's to stop a dual proposal submission?

- Identical proposals are sent to two completely different funding agencies, one or both of which may or may not have rules about overlapping proposals within each agency. Again, reviewers might balk at this, but is it wrong?

I have never attempted submission of overlapping proposals and am not really interested in doing so, though I'm not sure why not. There is something appealing about the general concept of sending a proposal out into a broad funding universe to see if anyone would like to give me money for my research. This would reduce the time spent trying to figure out to which one of several possible programs a proposal will be sent, a decision that may involve making likely unfounded assumptions about the scientific preferences and sanity levels of the program officers and hoping that an interdisciplinary proposal won't fall through the cracks between programs.

It does seem wrong, though. And speaking as someone who just reviewed quite a few proposals, I certainly don't want to see multiple versions of any of these, even the ones that were Very Good*.

* In NSF-speak, Very Good is not as good as Excellent but much better than Good, which is not good at all, except when compared to the dreaded Fair and Poor.

Proposed Ethical Lapse

These difficult economic times have resulted in many necessary cost-saving measures and cuts. These difficult economic times have also resulted in some unnecessary measures and cuts that do not save anyone any money. Examples of each:

• Freezing faculty salaries (and hiring) saves a university money.

• Freezing postdoctoral salaries (and hiring) does not save the university money if the postdocs are entirely paid from external sources of funding.

In fact, universities benefit financially from postdoctoral scholars because postdoctoral salaries may be part of the indirect cost calculation of a grant. Postdocs in the sciences bring money to a university. Freezing salaries of postdocs or other soft-money researchers is a money losing policy.

Perhaps administrators aren't aware of this? Perhaps they are aware but it's too difficult to make different policies for different job categories? Perhaps they are concerned about fairness? That is, why should one group of employees get a raise when others are experiencing salary freezes or cuts?

I know that faculty at various institutions have written letters to the powers-that-be about restrictive postdoctoral hiring/salary policies, with only limited success at overturning the policy or being granted exemptions.

I can't think of a good reason why grant-funded salaries can't be paid as budgeted in the grants. If the money exists in a grant for the specific purpose of paying a researcher, the researcher should get the budgeted money no matter what the university policy is regarding hiring/pay for faculty or staff.

Some of my colleagues have been trying to find ways to get around the salary freeze so they can pay their postdocs (or themselves) the amounts budgeted in grants. Here are some of the possibilities that I have heard discussed:

1. Writing and back-dating a letter promising a specific raise in an official letter to a postdoc. Although the salary for the first year is always mentioned in an offer letter/hiring contract, in some cases the salary for subsequent years is not. The proposed back-dated letter is an attempt to get around the lack of a specific letter for the second or subsequent years. If a pre-economic-crisis letter exists and spells out the salary/raise for subsequent years, the university has to honor this.

2. Finding a way to get the raise money to its intended recipient in a non-salary kind of way, e.g. buying an awesome personal computer of equivalent cost as the unpayable raise. Computer purchases can easily be justified on most grants. This isn't as good as a real raise, but it's better than nothing.

Back-dating a letter clearly isn't ethical, and I suppose spending grant money on something that wasn't originally budgeted and that might not directly impact the research isn't ethical either, but can these ethical lapses be forgiven because they are done for a good cause? Are the lapses justifiable because they are done to counteract a misguided policy, or should we follow the rules, however stupid they are?

If we can't spend the budgeted money on salary, we either have to spend the money on something else (something we are told at ethics training workshops is not allowed) or we have to give the money back to the funding agency at the expiration of the grant. Perhaps we can prolong our grants with no-cost extensions until the no-raise policy is lifted and eventually give our postdocs the budgeted raise, assuming the postdocs haven't moved on to another job.

I have recently considered another not-so-ethical route to take so that I can give a raise to a postdoc. The raise is in the grant budget, was justified in the grant proposal, and the postdoc deserves the raise. I made a request to the Dean that I be allowed to give the postdoc the budgeted raise, and my request was denied. So I started thinking of ways I could somehow get the raise to the postdoc anyway.

I could perhaps be talked out of my unethical idea by persuasive comments to this post, but at the moment this idea seems kind of appealing to me. Consider:

One of the only ways to be granted an official exemption to the no-raise policy is if the person in question has another job offer. The job offer doesn't have to be carved in stone -- it can just be an email from someone at another institution expressing an intention to offer a position. I don't want my postdoc to go out and get a real job offer (and he has said he wants to stay on here as a postdoc for another year or two), but I am pretty sure that I could get a colleague at another institution to send my postdoc an email expressing an interest in hiring him away from my institution (but without any real intention of doing so). With such a letter in hand, there's a good chance I could get the raise approved.

Ethical? No.. Should I do it anyway? Is there another, better way?

Co-authorial Rights

Something caught my eye in a recent NYT article about a supposedly falsified biomed study in which the first author forged the signatures of co-authors whom he selected without informing them about the study or their co-authorship. When problems with the (published) paper were revealed and the co-authors realized they were co-authors on a paper they had never seen, involving research they had not done, at least one of them tried to gain access to the reviews of the manuscript and other editorial correspondence with the first author. He was denied access.

This is from the article, with additional info added by me in brackets:

Dr. Andersen [one of the co-authors who didn't know he was a co-author until the paper was published], curious about what Dr. Kuklo [the first author] had actually submitted, asked Dr. Heckman [a journal editor] for copies of those reviews. But the editor turned him down, even though Dr. Andersen was supposedly one of the study’s authors. In a recent interview, Dr. Heckman said that his journal, like many others, considered such reviews confidential and shared them only with a study’s lead author.

“It is all confidential information,” Dr. Heckman said, when asked by a reporter for the reviews. “It is protected by the peer-review process.”

I can see why a random person couldn't write to a journal editor and request to see so-and-so's reviews, but why is this information confidential with respect to co-authors?

This general issue reminded me of a conversation I had at a conference last spring with a colleague. He hates the fact that he almost never gets to see the reviews and editor comments for manuscripts to which he has contributed as a co-author. Some of his first-author colleagues won't send him the reviews even when he asks. He had never tried asking an editor if he could see the reviews as well, but perhaps it wouldn't have mattered.

Is Dr. Heckman right? I am not sure he is, but if he is, why can't co-authors see review materials? If co-authors are responsible for the content of papers, shouldn't they have the right to see the reviews?

In a few instances, I (in my role as first author) have not wanted to share review and editorial correspondence with co-authors, for reasons I will outline below, but if any of these co-authors had asked me directly if they could see the reviews, I would have complied with their request. And if a journal had a policy of giving co-authors access to reviews, I would not object.

The exceptions I can think of at the moment occurred when:

(1) the co-author was a somewhat junior student and the reviews were 'not constructive' (= hostile and unprofessional, possibly including insulting comments). Eventually students should see reviews in their raw form and learn how to deal with negative comments displaying various magnitudes of rudeness; I expect senior grad students to participate fully in reading and responding to reviews. However, I have seen the crippling effects of harsh first-reviews on students, and would prefer to ease them into the experience of being attacked for no obvious good reason.

(2) I hated my co-author. I can think of one case in which I ended up not having much choice but to co-author a paper with someone who was not only hated by me but by most of the rest of the world. My other option was not submitting the manuscript, but I had a lot of time and effort (and $$) invested in this project and was unwilling to drop it without at least attempting to publish one paper. Communicating with my odious co-author about anything, however benign a topic, tended to unleash paranoid rantings about all the people he hated (they were wrong about everything, he was right), and I didn't want to know what his response would be to reviewer comments, even though they were mostly mild and constructive. I took care of all the revisions myself and presented the finished manuscript to him as a fait accompli.

Despite my aberrant and hypocritical behavior in these cases, my general opinion is that co-authors have a right to see reviews and be fully informed of the review process and editor decisions.

I realize that some manuscripts have 57 authors and it might not be practical to involve everyone, but perhaps in these cases the corresponding author could indicate the 5 or 10 co-authors who should have access, if they so choose, to information related to the review process.

Or am I missing something? Is there a downside to allowing some or all co-authors to have access to reviews?

Academic Spying

The New York Times has a front-page article today about couples spying on each other's email, webpage-viewing history, and cell phone use. This is non-news to most people, but what about academic e-spying?

The most serious incident I've had to deal with involved a postdoc's reading (and making copies of) my email and other computer files, but that was 7-8 years ago -- before computer security was taken as seriously as it is today. The postdoc (referred to in previous posts as 'the criminally insane postdoc') was fired, but because of that e-spying episode and another incident, my feelings towards internet communications can be fairly described as 'paranoid'.

I am not extreme about it -- for example, I am fond of internet shopping, and dozens of internet commerce sites have my credit card numbers stored on them. Furthermore, if I were completely paranoid, I probably wouldn't have this blog. When it comes to using the internet to communicate, though, I am less trusting. I suppose this is good in a way because it would discourage me from writing rude and salacious things in email messages, were I so inclined.

Some of my colleagues worry about internet security when sending or receiving reference letters. I have had a few students read paper copies of reference letters I had written for them (in one case this involved the student's opening a sealed envelope with my signature across the flap). And, as I've written before, sometimes people are shown their reference letters for tenure or promotion even when the letters are supposed to be confidential. Therefore, letters on paper are not necessarily more secure than electronic letters.

There are enough ways for academics to 'steal' (borrow) ideas and data without resorting to e-spying, so I imagine that such activities are mostly the realm of the crazies and the maliciously paranoid. But I could be wrong..

It's About Me

Is it OK (ethical) to help write a letter of reference for yourself? (for example, for jobs, promotions, awards)? From what I've seen, these situations are not uncommon. Perhaps I am ethically challenged in this respect, but I think it is OK if a candidate supplies information/text for a letter, as long as the ultimate letter writer agrees with the information and supplies additional information/text.?

Pay-Per-Review

Now that the academic year is over, it seems that reviewers and editors (including FSP) are getting caught up with their tasks. Yesterday I wrote about some reviews I received. Soon after, I got an email from someone who was reviewing another manuscript of mine.

I hate it when this person reviews my manuscripts because he always writes to tell me that he is doing the review, has some possible problems with my interpretations, suggests that we 'discuss' it, and then he asks me for a favor. Once he asked me to pay for his travel to the U.S. This time he asked me to do some analyses for him. Sometimes I request that this person not review my manuscripts, and I give a vague reason like "conflict of interest", though I didn't do that with this manuscript.

This particular unethical reviewer I've just described is the only one I've encountered who does this. I think it is a rare situation, but it raises the general and more relevant issue of how and whether to suggest that someone not review your manuscript or proposal.

Most journals (and NSF) give authors an option of listing "non-preferred reviewers". I don't like to use this option if at all possible. In an ideal world, we are all objective and will focus only on the Science. But back in the real world, there are people who should not review certain (or any) papers/proposals, and sometimes editors and program directors don't know who these people are without some information. If a reason must be given, "conflict of interest" is vague but professional. Somehow I have never felt comfortable writing things like "He harassed my postdoc" or "He asked me for money the last time he reviewed one of my papers".

If I don't have direct experience with someone's giving me an unfair review, I don't list them as a non-preferred reviewer. I would never use a suspicion that someone might be unfair as a reason to request that they not review something of mine. Delusional though it may be, it's best to give someone the benefit of the doubt unless there is evidence to the contrary.

As an editor, when I see non-preferred reviewers listed, sometimes I know the situation and can evaluate whether the list is valid, but in other cases I wonder why. I typically respect the wishes of the authors unless their list is long and includes everyone with any expertise in their subject. In these cases, the list of preferred reviewers is typically loaded with the names of the authors' friends and former co-authors. [memo to authors: Don't do this.]

I am sometimes asked whether it "looks bad" if you list non-preferred reviewers, as if you are afraid of criticism and not confident about the excellence of your paper. The advice I give is basically what I described above: if you have a concrete reason for requesting that someone be excluded from reviewing your work, you should do it. If you just have a suspicion, don't do it.

Ethics Overload (again)

Today I underwent more ethics training, and I fear that it was as irrelevant as all the other ethics workshops I have attended. At some point, one of the other trainees asked "Is there a point to this workshop?" and the person leading the workshop replied "Not really. We just have to do this. Be sure to sign the list so you get credit." Is that ethical?

Some of the workshop participants seemed to confuse ethical issues with issues of good vs. bad practices in science. Some people like to tell their own strange anecdotes that illuminate nothing (perhaps they should start blogs). Some people just like to disagree with whatever else is being said. It was a long afternoon.

The only time the workshop participants (including me) showed any signs of life was during a discussion of advisor-student issues re. publications. Someone told a supposedly shocking anecdote about an advisor who thought his students should be motivated to publish by the advisor's imminent tenure review. The workshop divided up into camps over that.

One faction's philosophy was: when you're in a research group, everything is interconnected. If someone isn't being productive, it affects everyone. If you're supported on a grant, you have to produce something or it affects the group's ability to get more grants.

Another faction thought that the advisor was selfish to focus on his tenure situation and maybe the students weren't ready to publish their results yet. It's hard to say without more information about that specific situation, but in general I relate more to research-group-as-interconnected-community point of view.

Too Close For Reviewing Comfort?

Today a colleague with whom I've collaborated at various times over the years told me he was asked to review one of my manuscripts on a topic completely separate from anything we've worked on together. He declined to review it because he felt it would be a conflict of interest given that we've collaborated on other research projects. A few months ago I wrote about a situation in which I didn't feel I could give an objective review of a proposal, and there certainly are situations in which lack of objectivity requires that one refuse to review a paper or proposal.

I must admit, though, that I have mixed feelings about this situation. I realize that one interpretation of my colleague's refusal to do the review is that he really doesn't like the work and didn't want to be involved in a negative review of a longtime colleague, but I don't believe that is the case here. I think he would give the manuscript a thorough, thoughtful, and overall positive review. And even if he had negative comments and was worried about how I'd react to them, he doesn't have to sign his review, so there would be no risk of my taking any negative comments personally.

I have seen many close and semi-close colleagues review each other's papers. In some of those cases I didn't think that was appropriate, but if we all declined to review the manuscripts and proposals of people we've worked with at one point, the reviewing system would collapse. That, or we'd only get reviews from people who disagree with us. So, where to draw the line?

The colleague who declined to review my manuscript is a very honest and sincere person, so I respect his decision. Part of me can't help feeling regret, though, that the 'old boy' network never seems to work for me.

Ethics Overload

My university has had some Issues over the years with ethical lapses of one sort or another, mostly involving biomedical faculty and researchers. Their occasional lapses result in an ever-changing set of university-wide requirements for ethics training for all faculty, researchers, and grad students.

I didn't mind this at first, and even volunteered to lead some of the ethics training sessions for grad students, as it was clear that there were some ethical issues that needed to be discussed with students: plagiarism of course, but also issues like authorship, data ownership, lab safety, etc. I also wanted the students to have access to resources for help with problems that might arise, or at least to know where to look for resources.

Faculty were initially required to attend 6 hours of workshops in ethics training. These workshops were completely useless. The people running them were not faculty, and had no experience with managing grants or people or labs or anything relevant to the topics being discussed. This requirement has gradually been expanded. There are some training activities we have to participate in every year, others every 3 years; some are online activities, some are in-person activities. All have been useless in a practical sense, and only have value in reminding us to take ethical issues seriously. If we don't get checked off for all the required ethics training activities, we aren't allowed to have grants.

If I were perfect and 100% in compliance with ethics requirements, I would have 4 different printers in my office, or I would have one printer with 4 different sets of ink cartridges and 4 sets of paper, and I would only print items for a particular project with that project's designated printer or printing materials. I would have all my office supplies designated by project as well, and would never use the X Project pen to write something related to the Y Project. I am not making up an obviously absurd situation -- this is what faculty were advised to do in one of the ethics sessions I attended. The person leading the session admitted under torture that she didn't really expect us to do this, but that she had to tell us to do it so that her responsibilities were covered and she wouldn't be to blame if we were busted by our funding agencies for using the wrong pen or printing supplies.

I am not a biologist; my scientific research does not involve human or animal subjects or their tissues or bodily fluids or thoughts or cultures or anything like that. Yet I am supposed to educate myself about policies regarding the Acquisition, Use, and Disposal of Human Bodies, just as an example. Is this a good use of my time? In fact (confession of ethical lapse!), I don't read those things. Instead, I peruse some of the nifty relevant things in the ethics training documents, like that I cannot use my NSF travel money to pay the difference between an economy airfare and a higher-class airfare. It wouldn't have occurred to me to do that, but I suppose there are those who gave it a try. So I check off the box that says I have read everything, but I do not print out the form to remind myself of all the important things I have learned. I don't know which printer to use.

Perhaps as more people acquire expertise with ethical issues in science and academia, the training activities can be better tailored for specific disciplines, but for now, these one-size-fits-all requirements and enforced workshops with talking head administrators seem to be generating widespread faculty contempt and loathing for ethics training requirements.