Commit 94ff3cab authored by Adrien Leger's avatar Adrien Leger
Browse files

update 2015_04_29_Improve_methodology_retraite_AGTI

parent 780cc869
......@@ -3,3 +3,4 @@
*.avi
*.mpeg
*.xvid
*.pdf
This diff is collapsed.
%% Cell type:markdown id: tags:
# PREPARE RISE PRESENTATION
%% Cell type:code id: tags:
``` python
from IPython.html.services.config import ConfigManager
from IPython.utils.path import locate_profile
cm = ConfigManager(profile_dir=locate_profile(get_ipython().profile))
cm.update('livereveal', {
'theme': 'solarized',
'transition': 'linear',
'start_slideshow_at': 'selected'})
# transition = default/cube/page/concave/zoom/linear/none
# theme = beige/blood/default/moon/night/serif/simple/sky/solarized
# start_slideshow_at = selected/beginning
```
%%%% Output: execute_result
{'start_slideshow_at': 'selected',
'theme': 'solarized',
'transition': 'linear'}
%% Cell type:code id: tags:
``` python
from IPython.display import HTML, Image, SVG, YouTubeVideo
```
%% Cell type:markdown id: tags:
# PRESENTATION STARTS BELOW
%% Cell type:markdown id: tags:
#Can we improve our methodology ?
### Adrien Leger
### Emilie Lecomte
### Jean-Baptiste Dupont
### Célia Couzinié
%% Cell type:markdown id: tags:
**Sure we can!**
%% Cell type:markdown id: tags:
#Why we have to improve our methodologies, analyses and reporting
![](http://static.blog.mendeley.com/wp-content/uploads/2012/08/Reproducibility-Initiative_resized.png)
**Emilie Lecomte / Jean-Baptiste Dupont / Célia Couzinié / Adrien Leger**
**Retraite AGT / 29 Avril 2015**
%% Cell type:markdown id: tags:
#Reproducibility crisis in science
#Believe it or not but Science is in a reproducibility crisis
Florian Prinz et al (2011), *How much can we rely on published data on potential drug targets?* Nature Reviews Drug Discovery 10, 712
Believe it or not: how much can we rely on published data on potential drug targets?
Editorial board (2012). *Must try harder*. Nature 483: 509–509.
Florian Prinz, Thomas Schlange & Khusru Asadullah
Landis et al. (2012). *A call for transparent reporting to optimize the predictive value of preclinical research.* Nature 490: 187–191.
%% Cell type:markdown id: tags:
Nature Reviews Drug Discovery 10, 712 (September 2011)
### I have numbers...
%% Cell type:markdown id: tags:
![](http://www.nature.com.gate2.inist.fr/nrd/journal/v10/n9/images/nrd3439-c1-f1.jpg)
%% Cell type:code id: tags:
%% Cell type:markdown id: tags:
``` python
```
###More numbers ...
%% Cell type:markdown id: tags:
bla bla
%% Cell type:markdown id: tags:
###And Nature group created the open acces collection
### "[Challenges in irreproducible research](http://www.nature.com.gate2.inist.fr/nature/focus/reproducibility/index.html)"
![Design](files/fig/nature.png)
%% Cell type:markdown id: tags:
#What are the consequences ?
%% Cell type:markdown id: tags:
* Biotech failure
* ...
%% Cell type:markdown id: tags:
###Biotech and Big Pharma loose confidence in academics
![](http://www.addconsortium.org/EVENTS/EV1000165.jpg)
%% Cell type:markdown id: tags:
#What caused the reproducibility crisis?
%% Cell type:markdown id: tags:
### Scientific fraud
** Real issue but concerns only an small proportion of irreproductible studies**
![](http://languagelog.ldc.upenn.edu/myl/NoahsPublisher.gif)
%% Cell type:markdown id: tags:
### Unrelated data panels / missing references / incorrect controls
=> big classic
%% Cell type:markdown id: tags:
### Pseudo replicates
Failure to understand the difference between technical and biological replicates =
%% Cell type:markdown id: tags:
### Underpowered studies = Small effect sizes
There may be an significant difference between 2 experimental groups but the study is underpowered and the conclusion indicate an absence of difference ...
Example Garcia 1 dog...
%% Cell type:markdown id: tags:
### Data handling
* Interim data analysis
* Ad hoc exclusion of data
%% Cell type:markdown id: tags:
### Misusage and Over-interpretation of statistical tests
Nuzzo, R (2014). *Scientific method: Statistical errors*. Nature 506: 150–152.
![](http://www.nature.com.gate2.inist.fr/polopoly_fs/7.15481.1392118822!/image/pvalue_art.jpg_gen/derivatives/landscape_630/pvalue_art.jpg)
%% Cell type:markdown id: tags:
### Inaccurate and incomplete reporting of methods
Impossibility to reproduce results example = experiment acinar Ed
**=> Report the methodology extensively, Provide raw tables in sup data in addition to graphics**
%% Cell type:markdown id: tags:
#How do we fix it?
%% Cell type:markdown id: tags:
### Formal training in statistics and study design of project leader
### Or
###work with consultants in methodology design and statistics
%% Cell type:markdown id: tags:
**Pas le temps! Pas l'argent! ...**
![](http://38.media.tumblr.com/df91e5cf7e1bfe9f8568b8e1a5a471f3/tumblr_nld3boY3Il1s8pd5uo1_500.gif)
%% Cell type:markdown id: tags:
**Study have to be designed before starting new projects**
**Exclusion criteria for all experimental protocols**
**
%% Cell type:markdown id: tags:
### Small sample effects
* Randomisation of animals
* For advanced preclinical studies = Reduce the number of experiental conditions
* Be more stringent in statistical tests = don't choose a statistical test because it looks better with your data
%% Cell type:markdown id: tags:
Standardization
Blinding
%% Cell type:markdown id: tags:
![](http://38.media.tumblr.com/d165f749635c27b7500f8adf2f37628a/tumblr_nl8zsrgLWw1s8pd5uo1_400.gif)
......
%% Cell type:markdown id: tags:
# PREPARE RISE PRESENTATION
%% Cell type:code id: tags:
``` python
from IPython.html.services.config import ConfigManager
from IPython.utils.path import locate_profile
cm = ConfigManager(profile_dir=locate_profile(get_ipython().profile))
cm.update('livereveal', {
'theme': 'solarized',
'transition': 'linear',
'start_slideshow_at': 'selected'})
# transition = default/cube/page/concave/zoom/linear/none
# theme = beige/blood/default/moon/night/serif/simple/sky/solarized
# start_slideshow_at = selected/beginning
```
%%%% Output: execute_result
{'start_slideshow_at': 'selected',
'theme': 'solarized',
'transition': 'linear'}
%% Cell type:code id: tags:
``` python
from IPython.display import HTML, Image, SVG, YouTubeVideo
```
%% Cell type:markdown id: tags:
# PRESENTATION STARTS BELOW
%% Cell type:markdown id: tags:
#Can we improve our methodology ?
### Adrien Leger
### Emilie Lecomte
### Jean-Baptiste Dupont
### Célia Couzinié
%% Cell type:markdown id: tags:
**Sure we can!**
%% Cell type:markdown id: tags:
#Why we have to improve our methodologies, analyses and reporting
![](http://static.blog.mendeley.com/wp-content/uploads/2012/08/Reproducibility-Initiative_resized.png)
**Emilie Lecomte / Jean-Baptiste Dupont / Célia Couzinié / Adrien Leger**
**Retraite AGT / 29 Avril 2015**
%% Cell type:markdown id: tags:
#Reproducibility crisis in science
#Believe it or not but Science is in a reproducibility crisis
Florian Prinz et al (2011), *How much can we rely on published data on potential drug targets?* Nature Reviews Drug Discovery 10, 712
Believe it or not: how much can we rely on published data on potential drug targets?
Editorial board (2012). *Must try harder*. Nature 483: 509–509.
Florian Prinz, Thomas Schlange & Khusru Asadullah
Landis et al. (2012). *A call for transparent reporting to optimize the predictive value of preclinical research.* Nature 490: 187–191.
%% Cell type:markdown id: tags:
Nature Reviews Drug Discovery 10, 712 (September 2011)
### I have numbers...
%% Cell type:markdown id: tags:
![](http://www.nature.com.gate2.inist.fr/nrd/journal/v10/n9/images/nrd3439-c1-f1.jpg)
%% Cell type:code id: tags:
%% Cell type:markdown id: tags:
``` python
```
###More numbers ...
%% Cell type:markdown id: tags:
bla bla
%% Cell type:markdown id: tags:
###And Nature group created the open acces collection
### "[Challenges in irreproducible research](http://www.nature.com.gate2.inist.fr/nature/focus/reproducibility/index.html)"
![Design](files/fig/nature.png)
%% Cell type:markdown id: tags:
#What are the consequences ?
%% Cell type:markdown id: tags:
* Biotech failure
* ...
%% Cell type:markdown id: tags:
###Biotech and Big Pharma loose confidence in academics
![](http://www.addconsortium.org/EVENTS/EV1000165.jpg)
%% Cell type:markdown id: tags:
#What caused the reproducibility crisis?
%% Cell type:markdown id: tags:
### Scientific fraud
** Real issue but concerns only an small proportion of irreproductible studies**
![](http://languagelog.ldc.upenn.edu/myl/NoahsPublisher.gif)
%% Cell type:markdown id: tags:
### Unrelated data panels / missing references / incorrect controls
=> big classic
%% Cell type:markdown id: tags:
### Pseudo replicates
Failure to understand the difference between technical and biological replicates =
%% Cell type:markdown id: tags:
### Underpowered studies = Small effect sizes
There may be an significant difference between 2 experimental groups but the study is underpowered and the conclusion indicate an absence of difference ...
Example Garcia 1 dog...
%% Cell type:markdown id: tags:
### Data handling
* Interim data analysis
* Ad hoc exclusion of data
%% Cell type:markdown id: tags:
### Misusage and Over-interpretation of statistical tests
Nuzzo, R (2014). *Scientific method: Statistical errors*. Nature 506: 150–152.
![](http://www.nature.com.gate2.inist.fr/polopoly_fs/7.15481.1392118822!/image/pvalue_art.jpg_gen/derivatives/landscape_630/pvalue_art.jpg)
%% Cell type:markdown id: tags:
### Inaccurate and incomplete reporting of methods
Impossibility to reproduce results example = experiment acinar Ed
**=> Report the methodology extensively, Provide raw tables in sup data in addition to graphics**
%% Cell type:markdown id: tags:
#How do we fix it?
%% Cell type:markdown id: tags:
### Formal training in statistics and study design of project leader
### Or
###work with consultants in methodology design and statistics
%% Cell type:markdown id: tags:
**Pas le temps! Pas l'argent! ...**
![](http://38.media.tumblr.com/df91e5cf7e1bfe9f8568b8e1a5a471f3/tumblr_nld3boY3Il1s8pd5uo1_500.gif)
%% Cell type:markdown id: tags:
**Study have to be designed before starting new projects**
**Exclusion criteria for all experimental protocols**
**
%% Cell type:markdown id: tags:
### Small sample effects
* Randomisation of animals
* For advanced preclinical studies = Reduce the number of experiental conditions
* Be more stringent in statistical tests = don't choose a statistical test because it looks better with your data
%% Cell type:markdown id: tags:
Standardization
Blinding
%% Cell type:markdown id: tags:
![](http://38.media.tumblr.com/d165f749635c27b7500f8adf2f37628a/tumblr_nl8zsrgLWw1s8pd5uo1_400.gif)
......
/*Some css fixes to use night theme with IPython slides*/
body {
background-color: #111111;
}
.reveal p {
color: white;
}
.reveal ul, ol {
color: white;
}
div.output_area pre {
color: white;
}
div.output_stderr {
background: #F00;
}
.reveal table th, .reveal table td {
color: white;
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment