Wednesday, August 20, 2014

A paper of mine was published. Did anyone read it?

An orthopedist asked me if I could explain why a couple of papers of his did not generate any feedback. He wasn't even sure that anyone had read them. He enclosed PDFs for me.

Not being an orthopedist, I cannot comment on their validity.

But I think I can explain why the papers have not created much interest.

Are you familiar with the term "impact factor"? If not, here is a link explaining what it is:

A journal's impact factor is an indication of how widely cited its articles are. One can also assume that it is a good indication of how popular the journal is and by inference, how many people read its papers. The impact factor has been criticized, but it is one of the few measures of a journal's influence.

The two papers in question were published in Orthopaedics & Traumatology: Surgery & Research. A list of the top 40 orthopedic journals ranked by impact factor in 2013 showed that it ranked 37th with an impact factor of 1.061. That means the average number of citations for any paper published in OTSR was about 1, and 36 orthopedic journals were more widely cited than OTSR.

A paper in Physics World claims that that 90% of published papers are never cited and 50% are never read by anyone but the authors and the journals' peer reviewers. I believe this is true of papers in medical journals too.

I was unable to obtain any figures regarding the number of subscribers to OTSR, but I suspect it is not large. This may also account for the lack of responses to the papers. My own experience is similar. It was very rare to receive any feedback about any of the over 90 peer-reviewed papers, editorials, or reviews that I had written.

Consider this. A blog post of mine "Appendicitis: Diagnosis, CT Scans and Reality" which I wrote 4 years ago has received over 19,600 page views and more than 100 comments. I am certain that post has been read far more than all of my published research papers combined. In fact, my 550 blog posts have recorded over 1 million page views.

What does it all mean?

Journals may have to adapt and become more like blogs. In the future, medical information may be disseminated by blogs and comments rather than journal articles and letters to the editor.

Will scientists' CVs be valued more for the number of page views their papers receive than the number of peer-reviewed papers they publish?

Monday, August 18, 2014

Are surgeons responsible for everything that happens to their patients?

Several months ago, a post called "Everything's my fault: How a surgeon says I'm sorry" appeared on KevinMD. It was written by a plastic surgeon who feels that no matter goes wrong with a patient, surgeons should never blame anyone else.

She gave some examples such as the lab losing a specimen, a chest x-ray that was ordered and not done, a patient eating something when he was not to be fed, and a surgeon having to cancel a case because the patient's blood pressure was elevated. She felt that all of these incidents should be owned by the patient's surgeon.

I agree that if I order a chest x-ray and find out later it wasn't done, I would accept the responsibility to have made sure it was done. I have always believed that you should not order a test if you aren't going to do something with the result.

But if the lab loses a specimen or a patient who was not supposed to eat does so, how is that my fault?

Wednesday, August 13, 2014

Applicant worries about the future of surgery

On my "Ask Skeptical Scalpel" blog, a medical student who is applying to general surgery programs is concerned about "The rise of the two M's—machines and mid-levels."

I talk him off the ledge. You can read the post here.

Monday, August 11, 2014

What can be done about letters of recommendation?

Many surgical educators feel that letters of recommendation are not particularly helpful in evaluating applicants or predicting eventual resident performance.

Among the issues are lack of uniformity of content, excessive use of superlatives, reliability—if the writer is not known to the recipient—and more.

Even if the writers are well-known academic surgeons, the degree of their personal knowledge of the applicants is not always clear.

During an extensive Twitter discussion over the weekend, someone mentioned that in an attempt to deal with this problem, emergency medicine had developed a standardized letter of recommendation.

A recent paper from the EM Standardized Letter of Recommendation Task Force shows that there is still work to be done. From the abstract:

For the question on "global assessment," students were scored in the top 10% in 234 of 583 of applications (40.1%), and 485 of 583 (83.2%) of the applicants were ranked above the level of their peers. Similarly, >95% of all applicants were ranked in the top third compared to peers, for all but one section under "qualifications for emergency medicine."

I've written before that dean's letters are more like public relations press releases than accurate assessments of a student's performance. You will rarely find negative comments in them. But another recent paper by a group of psychiatrists found that The presence of any negative comments in the dean's letter yielded significant correlations with future problems. Further, those applicants with future major problems had significantly more negative comments in the dean's letter than did those with future minor problems. Other factors such as USMLE scores, failed courses, letters of recommendation, and interviewer ratings and comments did not predict future problems.

These problems are not new. A 1983 New England Journal opinion piece about recommendation letters entitled "Fantasy Land" is remarkable for its validity even today. Here are a few choice quotes.

It's a land where everyone is "a pleasure to work with," has "excellent initiative," is "enthusiastic and conscientious," and possesses and "above-average fund of knowledge."

No one is ever poor, fair, or average; they are all "very good" or "excellent."

The author, Dr. Richard B. Friedman, said letters of recommendation were useless and advocated doing away with them.

A brief JAMA essay by Dr. Henry Schneiderman in 1988 called for more openness in describing students but acknowledged that negative comments were often "the kiss of death."

He proposed a new system of categorizing medical student performance. Here are just a few examples.

@AmirGharferi suggested this:

"Dr.G, do you feel comfortable writing me a strong letter?"
"Ok, I'll find someone who is."

That works if the student is aware enough to ask, and the faculty member is honest enough to say no. In my experience, even the most marginal of students can find someone—in addition to the dean, of course—to write a good letter.

I am no longer involved in the process of selecting residents. I have no suggestions.

What is your solution to this problem?

Friday, August 8, 2014

True grit

In case you haven't noticed, a hot new topic in education is "grit." In order to reduce the long-standing 20% attrition rate of surgical residents, some say we should select applicants who have more grit or conscientiousness.

A recent paper in Surgery reported residents who dropped out of programs had decreased levels of grit as measured by a short-form survey. But due to unexpectedly low attrition rates in the surgical programs participating in the research, the study was underpowered to show a statistically significant difference in outcomes of high-vs. low-grit individuals.

It's hard to argue with the premise that choosing applicants with high reserves of grit might lead to better retention and performance of residents.

I blogged about this three years ago in a post called "Harvard says train residents and medical students like Navy SEALs."

Unfortunately, identifying who has grit will be much more difficult than simply testing those applying for surgical residency training.

Below is the eight-item grit survey, which is scored on a 1 to 5 Likert scale.

Tuesday, August 5, 2014

What to do with abnormal PSA results in a young man?

A 45-year-old man in excellent health with no family history of prostate cancer had a screening PSA done three years ago which was in the range of 4.0 ng/mL. He has been followed by a urologist, and the test was repeated several times without much change.

In June of this year, his PSA was 4.6 and the free PSA was 0.6 for a ratio of 0.13. He was given a course of antibiotics for presumed prostatitis, and repeat testing a month later showed a PSA of 3.8 with a free PSA of 0.5. Because the PSA was less than 4, a ratio was not calculated.

The patient obtained copies of the reports. The from June one states the following: "When total PSA is in the range of 4.0-10.0 ng/mL, a free PSA/total PSA ratio of less than or equal to 0.10 indicates a 49% to 65% risk of prostate cancer depending on age. A free PSA/total PSA ratio of greater than 0.25 indicates a 9% to 16% risk of prostate cancer depending on age." It does not comment on the significance of a ratio of 0.13, which I have looked up. The cancer risk is in the area of 20%-25%. However, no source gave estimates for men under the age of 50.

On examination, his urologist can feel no nodules. He has recommended that the patient undergo an MRI of the prostate.

Stating that an MRI is not indicated in a man of his age with his history, the patient's insurance company will not pay for the test and suggested a trans-rectal ultrasound. The urologist advised the patient not to have an ultrasound due to his age and the potential for complications. A hospital quoted him an out-of-pocket price of $2500 for the MRI.

The urologist has told him that random biopsies may not be accurate and there is a risk of complications.

When he had his first PSA done three years ago, I had expressed surprise and wondered why it had been offered to him. The patient said his internist told him he should have the test.

He is concerned about these recent results and has asked me for advice which I am not qualified to give.

What would you advise?

Follow-up August 6, 2014

Numerous urologists responded on Twitter with more than 60 tweets about this post. Suggestions for the next step were as follows: trans-rectal ultrasound (TRUS) and biopsy as mentioned by Dr. Cooperberg below; going ahead with the MRI; repeating the PSA in 3 months; go for a second opinion by a recognized expert in prostate cancer.

One urologist emailed me to point out that even if the patient has cancer, it is probably not an aggressive type because his PSA has not risen in 3 years.

Another urologist gave me the name of an expert in the patient's geographic area. 

The patient has already scheduled an appointment for a second opinion.

Board passage rates and residency program quality

On my "Ask Skeptical Scalpel" blog, a medical student wonders if a program's board passage rate is a good measure of whether its graduates can practice independently.

You can read that post here.