File Drawer Effect/Problem
The ‘file drawer effect/problem’, termed by Robert Rosenthal in 1979, is quite easy to define, it refers to the suggestion that some researchers file away and never publish studies that have not been able reject the null hypothesis. It suggests that only positive-outcome research are reported, and negative outcome research are filed away and forgotten. Negative outcome research not only includes research with no or little significance, but also research that may have found something else of significance but was not expected or goes against researchers hypothesis, all findings should be available, for it is essential for scientific progress. Other explanations for negative outcome research being filed away are; small studies which maybe seen as irrelevant, when effect size is small, if there are flaws in research, for all of which, common sense will imply that reliability and validity are questionable. Another factor for escalating this problem is when the research has much financial funding and interest, for which findings may have negative implication to the sponsor, or public well being is at risk (this could go anywhere from here if you are a believer in conspiracy theories……).
The worst case scenario of this problem is that published and reported research are the 5% that are type 1 errors if p level is set at .05, and the other insignificant 95% are hidden away in a filing cabinet, considering this extreme scenario and the fact that maybe some new findings are not reported and published, how can science possibly progress, we will never become closer to the truth. What is concerning, is that what is reported could be a misrepresentation of the subject under investigation, nevertheless bear in mind this is the worst case scenario possible.
This is especially problematic when considering meta analysis. If using data from previously performed research and integrating them for new analysis and research, how can this be valid, reliable and representative, if all that is analysed is significant biased results.
Another problematic aspect of the ‘file drawer effect’ could be that negative outcome research maybe filed away, but future research maybe tweaked if variables that caused insignificant findings were identified, and result in future significant findings. This links with Karl Poppers belief that researchers protect their hypothesis, and it is why he came up with the criterion of falsification (which is another topic for which I will go no further).
There has been little research into this problem. However, a study by Martinson et al in 2005, discovered that 6% of researchers admitted to having “tossed out data because the information contradicted their previous research.”, so therefore we must conclude that the problem does exist.
Unfortunatley there is not a definite solution to this problem, but there are suggestions to reduce and cope with the problem which are; better powered studies, where bias in meta-analysis is reduced; larger studies to enhance reliabililty; better research standards; assessment of research by trained professionals to reduce reasons for research being ‘filed away’. Many prominent journals now require that all research are registered before study takes place, and separate journals created for negative results, so therefore whether findings are significant or not there is a record of the research, it also enforces guideline for accurate reporting. This is hoped to reduce the ‘file drawer problem’, and will also indirectly address some of the previous mentioned methods to reduce the problem. Although research suggests that these initiatives have not been entirely successful, it is a start, and is better that the problem is addressed than ignored.
prpfe76 
It is shocking to think that 95% of studies are ‘hidden away’! The ‘keeping it under your hat’ is a well-known activity across scientific societies publically known as the file-drawer phenomenon.
The file-drawer effect in which only positive results tend to get reported and negative ones are left in the filing cabinet, and greatly biases any analysis of combined results (Radin, 1997). As you state, the file-drawer effect is very much problematic in meta-analysis, as meta-analysis must incorporate a procedure for taking the file-drawer effect into account. Radin, 1997 found that more than 3,300 unpublished, unsuccessful reports would be needed for each published report in order to “nullify” the statistical significance the p value.
The file-drawer effect is not limited to meta-analysis only, many studies are biased towards obtaining positive results, in particular clinical trials that are commercially sponsored; the trials with a more positive predicted outcome are more likely to have priority by the funding bodies. Clearly this means that a substantial amount of clinical research are never reported to the public domain because a treatment effect was not shown, these are what are referred to as ‘negative studies’. Having said this, in many cases, clinical trials are usually a commercial liability as they require huge amounts of money and investments in order to develop new drugs and for the results to be successful, which pressurizes companies to produce ‘positive studies’, and no wonder the file / drawer phenomenon happens!
The most evident in concealment of flagrant abuse is clinical trials of selective serotonin reuptake inhibitors in children. Seeing as depression is not an easy diagnosis to make in the emotional life of children and teenagers, by the age of 18, 20% of teenagers will have experienced an episode of major depressive disorder, usually lasting for a few months, and carrying the threat of recurrence. The large market of child depression attracts clinical and pharmaceutical companies to conduct studies on SSRIs. Jane Garland, a clinical researcher that specialises in SSRIs states that many of her studies have gone unpublished; she believes that in order to obtain a ‘positive study’ “never do an industry-funded trial again unless the whole structure and management of these is drastically changed”, maybe if all researchers took this approach perhaps the file-drawer effect would be recuced.
http://www.colorado.edu/philosophy/vstenger/Briefs/Meta.html