Conservapedia - User contributions [en]

Talk:Significance of E. Coli Evolution Experiments

2009-03-26T01:01:57Z

Argon: /* Unreferenced Claims */

SJohnson, your assessment, while good in the utilization of the chi-squared test is unfortunately incorrect. The Monte Carlo resampling gives a more accurate p-value than the chi-squared. You may research the literature (i.e. publications in statistical mathematics, many pubs actualy compare Monte Carlo vs Chi Squared) to discover that this method is commonly used in advance statistical work and how it is more accurate than the chi-squared test.--[[User:Able806|Able806]] 17:00, 4 March 2009 (EST)

:It doesn’t make sense to compare the chi-square test, which is a specific statistical hypothesis test, to Monte Carlo methods, which can be used for anything from fluid motion modeling to p-value computations. You can use Monte Carlo methods to compute the p-values of the chi-square test!

:Monte Carlo methods involve the generation of random realizations. Your broad claim the Monte Carlo methods are “more accurate” than the chi-square test is obviously incorrect because the accuracy of Monte Carlo methods always depends on the number of random realizations generated. When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous.

:Which publications compare Monte Carlo to chi-square and show that the former is more accurate? Could you provide specific examples? Thanks. [[User:SJohnson|SJohnson]] 18:50, 4 March 2009 (EST)

:In furtherance of SJohnson's remarks with respect to rarely occurring events, the use of the basic Monte Carlo method is plainly incorrect for modeling a rarely occurring event, as the Lenski paper did. This has long been pointed out in [[Flaws in Richard Lenski Study]]. I know [[evolutionists]] will never admit a flaw in anything promoting their pet theory, but this (and other) flaws in that paper is undeniable.

:Watch how evolutionists defended obvious errors in the Lenski paper, and then realize why the [[Piltdown Man]] fraud was taught for 40 years without evolutionists admitting it was a hoax.--[[User:Aschlafly|Andy Schlafly]] 09:55, 5 March 2009 (EST)

::Andy, how exactly is the Monte Carlo method incorrect to use in this case? I have seen it used in publications with much smaller datasets.--[[User:Able806|Able806]] 10:29, 5 March 2009 (EST)

:::Able806, I'm interested in looking at the publications you mentioned that use Monte Carlo methods to analyze small data sets. Could you provide some examples? Thanks. [[User:SJohnson|SJohnson]] 16:41, 5 March 2009 (EST)

::::SJohnson, here are two papers, [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WH8-45RFJ1J-19&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=1ad95954654bb97b17e474ce6b469f6e 1] and [http://cat.inist.fr/?aModele=afficheN&cpsidt=787963 2]. Most are in chemistry and genetics where you find the observed to be much smaller and have to use the MCM. You can search on the subject as well and find that how Lenski performed the test is the standard for microbiological genetic analysis.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

:::::Those papers have nothing to do with hypothesis testing. One is an archeology paper. To be blunt, it seems like you’re just doing internet searches on “Monte Carlo” to find these links. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

::::::SJohnson, actually they do, did you read the papers? If so you would see how they used the MCM for their data analysis of small data sets, which indeed was hypothesis testing and answers you inquiry about publications that use MCM for small data set analysis. If you wish I can try to track down some actual mathematical publications, however, I am not as familiar with mathematical journals as I am with science/medical journals (not knowing which mathematical journals are acceptable). I am assuming that you have a background in math and possibly access to mathematical journals, therefore if you know the reputable ones I can do the leg work.
::::::I believe the thing that needs to be looked at is there truly a problem with the choice of test and if so what is an alternative. Bayesian might be an option but seems to be difficult to employ for this situation.--[[User:Able806|Able806]] 12:36, 12 March 2009 (EDT)

:::Able806, you still seem to miss the point about how inappropriate the Monte Carlo method (as used in the Lenski paper) is for evaluating rarely occurring events. You need to open your mind to be productive. If you simply cling to a view that Lenski (who I don't think has any meaningful education in statistics) must somehow be right, then you're not going to make any progress in understanding the flaws.--[[User:Aschlafly|Andy Schlafly]] 17:07, 5 March 2009 (EST)

::::Andy, you still have not answered what you find inappropriate about his use of the Monte Carlo method? I am a reasonable person and with evidence I do have an open mind. I provided examples last week, with a working model, showing that Monte Carlo is better than the chi-square in this case. I have also shown where the Chi-Square was inappropriate due to the occurrence size as well. So if you have any evidence that Monte Carlo should not be used in the way that Lenski used please let it be shown.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

Sjohnson, I believe you just proved my point. In the literature of mean and covariance structure analysis, non-central chi-square distribution is commonly used to describe the behavior of the likelihood ratio statistic under alternative hypothesis; it is widely believed that the non-central chi-square distribution is justified by statistical theory. Actually, when the null hypothesis is not trivially violated, the non-central chi-square distribution cannot describe the LR statistic well even when data are normally distributed and the sample size is large. Monte Carlo results compare the strength of the normal distribution against that of the non-central chi-square distribution. In an association analysis comparing cases and controls with respect to allele frequencies at a highly polymorphic locus, a potential problem is that the conventional chi-squared test may not be valid for a large, sparse contingency table. Reliance on statistics with known asymptotic distribution is unnecessary, as Monte Carlo simulations can be performed to estimate the significance level of the test statistic.

Here is a [http://faculty.vassar.edu/lowry/chi_beta.html link] to a great page the provides an interactive example as to why the Chi Squared test would provide poor results compared to the Monte Carlo in relation to the Lenski data workup.

Something you may have overlooked was that the data set is actually too small to use the chi square method correctly. It is often accepted that is any of the analyzed data falls under 10 for a particular cell of the data set then the Yates correction needs to be applied; unfortunately the Yates correction can over correct thus skewing the p-value. Lenksi seemed to understand this by supporting his Monte Carlo p-value results with the Fisher z-transformation p-value.

I hope this helps.--[[User:Able806|Able806]] 10:27, 5 March 2009 (EST)

:I’m still waiting to hear which literature says that “Monte Carlo resampling” is “more accurate than the chi-squared test”. The page mentioned above [http://faculty.vassar.edu/lowry/chi_beta.html] is a discussion of why statisticians “fail to reject the null” rather than “accepting the null” when the p-value is above 0.05 or so. The page says nothing about superiority of Monte Carlo methods. Why were alternate hypothesis distributions mentioned? Only the null hypothesis distribution is used to calculate a p-value. Yates’s correction is for 2x2 contingency tables [http://en.wikipedia.org/wiki/Yates%27_correction_for_continuity]. It doesn’t apply in this case. Finally, what the heck do “covariance structure analysis” and “allele frequencies at a highly polymorphic locus” have to do with this problem? [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::SJohnson, I am looking for this paper for you, I cited it for one of my past publications dealing with allele frequencies (I believe it came from the Duke Biostatistics group). To answer your question about allele frequencies, that is the issue at hand, more about the genetics than the math, but it is the item being studied. So you stated that Yates can not be used and statistics says the number of occurrences is too small to evaluate using the Chi-Squared test so what would you recommend instead of the Monte-Carlo Method?

:Regarding the "Fisher z-transformation p-value" from the paper, garbage in garbage out. If the p-values were bad to begin with, then why would a combination of them be meaningful? [[User:SJohnson|SJohnson]] 10:49, 9 March 2009 (EDT)

::You are assuming that p-values are wrong based on a test that is inappropriate in this case due to data limitations. Did you perform a z-transformation on the chi-squared for the three data groups?--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

::You asked about the “Fisher z-transformation p-value”. The z-transformation test and Fisher’s method are actually two different things (see Whitlock's 2005 paper - Ref. 49 in Blount et al.). But no, I haven’t tried either. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::There's a large literature on various kinds of Monte Carlo test, a very short summary of which is that they're inevitably more accurate than parametric tests (e.g. F, t, chi-squared, etc) because they don't make assumptions about the distribution of the data under the null hypothesis. See for example ''Introduction to the Bootstrap'' by B. Efron and R. Tibshirani and ''The Jack-knife, the Bootstrap and Other Resampling Plans'', also by Efron. They're certainly applicable to small datasets and their accuracy is really only limited by the number of samples you care to take. E.g. 1000 M-C samples would give you a pretty accurate idea about significance at the alpha<1% level (That book should answer SJohnson's questions of 18:50 on 4/3/09 and 16:38 on 5/3/09 about accuracy and Aschalfly's comment of 17:07 on 5/3/09 about appropriateness of Monte Carlo tests.) [[User:FredFerguson|FredFerguson]] 16:53, 11 March 2009 (EDT)

::::Your claim that Monte Carlo methods are “inevitably more accurate” than other tests is obviously wrong because the accuracy of MC methods always depends on the number of realizations used. You should have written <math>\alpha=1\%</math>, not <math>\alpha<1\%</math>. If 1,000 random realizations are generated, the number of realizations above the true <math>\alpha=1\%</math> level is binomial with mean 10 and variance about 10. Thus, the standard deviation of the MC estimate is >0.003. In this example, a Monte Carlo p-value could be off by 30% and still be within a standard deviation. Is that really “pretty accurate”?

::::Using one million MC realizations (as done in the paper) at the <math>\alpha=0.001</math> level means the standard deviation is about 3%. The paper reported a p-value of less than 0.001 (experiment two). It wouldn’t surprise me to find out that the experiment two p-value for the flawed test is off because only one million realizations were used. My original statement, “When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous” is correct. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::::You're talking about miniscule differences in the accuracy of a test. 0.013 isn't very different from 0.007. In either case, it's very unlikely the experimenter would have obtained that result if the null hypothesis were true. If you're bothered about differences in P-values to the third decimals (which would make you unusual!), just run more MC realisations, that's all. Not really a problem. [[User:FredFerguson|FredFerguson]] 11:53, 12 March 2009 (EDT)

There’s still confusion about the difference between test statistics and Monte Carlo methods. Before you find a Monte Carlo estimate of a p-value, you need to select a test statistic to reduce the data set to a scalar. I am interested in hearing which test statistic you believe should be used in place of the chi-square test and why. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

----

Quick question for SJohnson: How many degrees of freedom did you choose when calculating the p-value? I'd like to know upon what condition you base that number. Thanks.--[[User:Argon|Argon]] 11:05, 5 March 2009 (EST)

:The degree of freedom for a contingency table is rows minus one times columns minus one. That is, <math> (r-1)(c-1) </math>. Here’s a pretty good tutorial I came across: [http://faculty.uncfsu.edu/dwallace/lesson%2020.pdf]. For the experiments from [http://www.pnas.org/content/105/23/7899.full.pdf], the DOFs are 11, 11, and 13. For experiment one, the chi-square test statistic is
::<math>
X^2
=\sum\limits_i\sum\limits_j
\frac{\left(n_{i,j}-E\left[n_{i,j}\right]\right)^2}
{E\left[n_{i,j}\right]}
</math>
::<math>
=\frac{\left(0-1/3\right)^2}{1/3}
+\frac{\left(6-17/3\right)^2}{17/3}
+\frac{\left(0-1/3\right)^2}{1/3}
+\ldots+
</math>
::<math>
+\frac{\left(2-1/3\right)^2}{1/3}
+\frac{\left(4-17/3\right)^2}{17/3}
</math>
::<math>
\approx
14.82
</math>
:where <math>n_{i,j}</math> is the observed value and <math>E\left[n_{i,j}\right]</math> is the expected null hypothesis value. So if you have MS Excel, another way to arrive at the p-value of 0.19 is to type “=CHIDIST(14.82,11)” into a cell. Cheers! [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::OK, thanks for the info. From what I'd calculated and looked up in tables, the numbers seemed close to a df=11 for a chi-square of ~14. (Aside: With terms having 17/3 in the denominator in the figures above, were you using the test of independence? I was using Pearson's test for [http://en.wikipedia.org/wiki/Pearson%27s_chi-square_test#Test_for_fit_of_a_distribution fit of a distribution] which returns a chi-squared value of 14 and roughly matched the p-values you reported, assuming the df was 11).

::Also, the first sentence of the article reads: "Blount, Borland, and Lenski[1] claimed that a key evolutionary innovation was observed during a laboratory experiment. That claim is false." A small correction: There were several claims in the paper. The 'key evolutionary innovation' was acquiring the ability to utilize citrate as a food source. That claim was demonstrated multiple times. The claim, which pertains to this statistics discussion was that the Cit+ phenotype arose in a multi-step process, first requiring a rare, pre-adaptive mutation before additional mutation(s) lead to the subsequent development of citrate utilization.--[[User:Argon|Argon]] 20:46, 5 March 2009 (EST)

:::My biology-degreed wife assures me that mutation does not necessarily mean that evolution occurred. What the paper claimed is that evolution (a “key innovation”) occurred in the lab. The key innovation supposedly increased the mutation rate. In the experiments, the observed mutation rate increased after generation 31,000, but not enough to make a statistically significant claim that the rate is not constant. The analysis in the paper was similar to flipping a coin ten times, counting six heads and claiming that the coin must be biased against tails. In reality, there’s nothing surprising about a fair coin producing slightly more of one outcome than the other. Just like there's nothing surprising about there being slightly more mutations in later generations than early generations given the null hypothesis (constant mutation rate). [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

>>Inserting a later comment first<<
SJohnson, the paper's title is: "Historical contingency '''and the evolution of a key innovation''' in an experimental population of ''Escherichia coli''" As I mentioned earlier, the key innovation is the evolution of the Cit+ phenotype and not the timing or rate of its acquisition. And yes, it *is* evolution (call it microevolution, if you wish). Blount et al went on further to speculate how this evolutionary innovation arose and they proposed the historical contingency hypothesis in which 'pre-adaptive' mutations were required before the Cit+ phenotype developed. It is only this latter hypothesis that you are attempting to address with your chi-square analysis, not the fact that Cit+ mutants arose (which is the evolutionary innovation).--[[User:Argon|Argon]] 21:57, 18 March 2009 (EDT)

::::SJohnson, not to say anything about your wife, but has she had a 400 level molecular genetics course (most general biology degrees do not cover the detail unless they are specialized)? If so, she would have mentioned that if the mutation passes to the offspring and is selectively beneficial to the population then it is a step of evolution as along as the conditions continue through the sharing of the mutation with the population and the environment is such that reduces the growth rate of the non-transformed population. While not all mutations are signs that evolution occurred the mutations that pass to offspring and provide a benefit compared to other offspring are very strong indicators. In the case of this paper the population that evolved the cit+ was able to metabolize a chemical in their environment which allowed for an adaptation advantage compared to the non-transformed colonies.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

----

Let’s go back to the beginning. There appears to be confusion about the difference between test statistics and methods for computing p-values. As is noted at the beginning of the page [http://www.conservapedia.com/Significance_of_E._Coli_Evolution_Experiments], the fundamental problem with the paper is that it used a flawed test statistic, not that it used Monte Carlo methods to find the p-value for that flawed statistic.

Every hypothesis test uses a test statistic to reduce the data to a single number. The p-value for the test statistic can be calculated analytically (as I’ve done for the chi-square test statistic) or by Monte Carlo methods. In the paper, Monte Carlo methods were used to compute the p-value of the “mutation generation” test statistic. The key problem with the analysis from the paper is that it doesn’t work to use a weighted average to test for variations in mutation rate. This is like trying to use the sample variance to test for an increase in the mean in Gaussian-distributed data. A statistic should be selected based on the null and alternate hypothesis distributions of the data. The chi-square test (unlike the weighted average from the paper) is a reasonable choice for data that mutates at a constant rate under the null hypothesis, but mutates at varying rates under the alternate hypothesis.

Able806, you made a good point about the contingency table cell frequencies being relatively low, but were wrong when you said ”the data set is actually too small to use the chi square method correctly”. In the low cell frequency case the chi-square test is still effective, but the null hypothesis distribution of the chi-square statistic starts to look less like the chi-square distribution. Thus, p-values calculated using the chi-square distribution may be a bit off. However, Monte Carlo p-values are always imperfect as well because it's impossible to generate an infinite number of random realizations. There are imperfections in p-values generated by analytic and Monte Carlo methods. However, low cell frequencies does not explain the >20x and >2.5x differences between chi-square p-values and p-values from the paper for experiments one and three. The reason for those huge differences was the use of the flawed test statistic (“mutation generation”) in the paper. [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

:SJohnson, the chi-squared test is a valuable statistical tool, but the limitations of the test must be acknowledged. The chi-squared test can only produce valid results if the assumptions that underly the test are not violated. As an analogy, Newtonian models of motion fail to produce accurate results as velocities approach the speed of light; under those circumstances one must switch to a theory that accounts for relativistic effects.

:It seems that you have simply dismissed the [http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm widely-acknowledged] [http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm fact] that the [http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html chi-squared test] is [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236 inappropriate] for use in [http://www.graphpad.com/www/Book/Choose.htm situations] where n in any cell is [http://mysite.du.edu/~jcalvert/econ/chisquar.htm less] less than a [http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1 threshold] [http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html number]. Different authors set different thresholds, but all are well above the numbers seen in your chi-squared analysis - even the most liberal guidelines advise against the chi-squared test when any expected cell frequency is less than one or more than 20% of the table cells are less than 5; others require that expected values in all cells must be more than 5. With smaller amounts of data, the test is insensitive and errs on the side of rejecting the hypothesis. If you attempt your chi-squared statistical analysis with a program that is more sophisticated than MS Excel (as I did), you get an error message indicating that the results are invalid due to low expected cell counts.

:That issue aside, there are other reasons that the chi-squared test is inappropriate here. As the links above point out, the categories tested must be truly independent; one example is that you can't use the chi-squared test to compare age and ability to kick a field goal by testing the same experimental group twice, one year apart; you have to test one group of age A and a different group of age B. In the case of the Blount paper, the categories are not independent. Even if there were adequate numbers to address the low-expected-frequency problem, this would make the chi-squared an invalid test in this case.

:There are other significant problems with the use of the chi-squared test in this circumstance, but they can wait until you address these first major problems.--[[User:ElyM|ElyM]] 12:18, 11 March 2009 (EDT)

::Wackerly et al. says in general it’s assumed that the cell frequencies are above five so that the chi-square statistic (under the null) is approximately chi-square distributed (see p. 703). That book does not say chi-square test results are invalid if frequencies are five or less. Your example of a chi-square test warning message (it said "warning" not "error" as you stated) in Minitab [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236] said “approximation probably invalid” referring to the chi-square distribution approximation to the chi-square test statistic’s distribution. Your example did not say “chi-square test invalid”. I agree that when cell frequencies are low, the chi-square test statistic’s distribution starts to deviate from the chi-square distribution. I maintain that this deviation is not enough to explain the >2.5x and >20x differences in the chi-square test p-values and the p-values from the paper.

::As the numerous links in your post proved, the chi-square test is widely-used by statisticians. Can you give examples of statisticians using mean mutation generation as a test statistic? Also, did your software agree with the chi-square test p-values I presented? Thanks. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::Thank you for giving page references for Wackerly; however it seems we have different editions, since page 703 in my copy (5th ed, 1996) does not deal with chi-squared issues at all. My copy does state the following, on page 622: "Although the mathematical proof is beyond the scope of this text, it can be shown that, when n is large [chi-squared] will possess approximately a chi-square probability distribution in repeated sampling." Then, on page 624: "Experience has shown that cell counts [n sub i] should not be too small in order that the chi-square distribution provide an accurate approximation to the distribution of [chi squared]. As a rule of thumb we require that all expected cell counts equal or exceed 5, although Cochran (1952) has noted that this value can be as low as 1 for some situations." Wackerly then goes on, in the problems sections, to describe the use of the chi-squared test as a "violation of good statistical practice" when "some expected counts [are] <5."

:::It seems that you are already aware that the [chi-square] statistic under the null is no longer chi-square distributed for small n; this is precisely why the test should not be used under those conditions. I can claim to be able to accelerate a 1-kg mass to 10 times the speed of light by applying 1 N of force for 95 years by using F=ma and t= (vf-vi)/a. Plugging the numbers into those equations will produce the same result every time, but the answer is illegitimate because those equations are only valid under certain assumptions, which are violated as velocities approach the speed of light. Similarly, having a statistical program calculate a chi-squared value given the Blount data will produce a number result, but since the assumptions of the test are violated the result is not legitimate. Yes, if I put the Blount data in SAS 9.2, I get the same numerical answer as you do, but I also get the following message: "WARNING: >89% of the cells have expected counts less than 5. Chi-square may not be a valid test." You may argue that that's a warning, not an error; that's a semantic distinction. The reason that the program says that it MAY not be valid is that the chi-squared test skews in the direction of being too conservative at low n values; the test has an acceptable rate of false positives but an unacceptably high rate of false negatives. Comparing the results of the Monte Carlo and chi-squared results in this case is like comparing the results of Newtonian and relativistic equations of motion: they can produce very different results from the same input data.

::::For a finite amount of data, the chi-square statistic is never chi-square distributed under the null. The p-values are always approximate regardless of cell frequencies. The approximation becomes more accurate as the amount of data increases, but I don’t believe that this inaccuracy will change p-values that are about 0.2 (for experiments 1 and 3) into statistically significant p-values. How much do you expect the p-values to change if an exact computation is used in place of the chi-square distribution approximation? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::Your last paragraph has a major non sequitur in it: yes, many statisticians use the chi-square test. As long as the assumptions of the test are not violated, it is a valuable tool. That has nothing to do with the validity of using mean mutation generation as a test statistic. 'Mean number of werewolf attacks in Mumbai in the week centered on the new moon, by month, from 1654 to 1798' is a valid test statistic. I am quite sure that it has never been used in a peer-reviewed paper before. That does not mean that I can't perform valid statistical tests on that statistic. If, however, the incorrect test is applied, the results of the analysis will be flawed. Papers apply a (relatively small) standard repertoire of valid tests to a (potentially infinite) number of test statistics. The particular test statistic used in a paper may never have been used before and may never be used again; that does not address the validity of the analysis. In Blount's case, the test is the Monte Carlo analysis, which is also "widely-used by statisticians".

::::There are an infinite number of ways to reduce a data set to a single number. However, it’s foolish to think every method would be effective. I gave an example of a flawed test statistic in an earlier post [http://www.conservapedia.com/index.php?title=Talk%3ASignificance_of_E._Coli_Evolution_Experiments&diff=635070&oldid=634987]. Another example of a flawed test statistic is the one used in the paper because it does not always detect deviations from the null hypothesis (see: [[Significance of E. Coli Evolution Experiments#Test Statistics]]).

::::Test statistics are typically derived. The likelihood ratio test is a common method used to derive them. The chi-square test for independence is an approximation to the LRT. Where is the derivation saying that mean mutation generation is an appropriate test statistic for this problem? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::We still haven't touched on the issue of the categories not being independent, which by itself is sufficient to invalidate the chi-squared technique. I'm new to this site, so I'm unsure as to the etiquette of making changes to the articles of another person - but the article here should at the very least mention that the chi-square test is being used here in a manner that violates its underlying assumptions in at least two fundamental ways, and the results are therefore suspect.--[[User:ElyM|ElyM]] 17:34, 12 March 2009 (EDT)

:::::When generating random realizations of experiment outcomes, the authors assumed that the total number of mutants was fixed. Thus the paper assumed the numbers of mutants per generation are statistically dependent. Does this seem like a realistic model, or do you think that if the experiments were recreated that the total number of mutants could vary? For example, if experiment one were recreated, would the total number of mutants always be exactly four? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::: It looks to me as though SJohnson has misinterpreted the application of the chi-squared test in quite a fundamental way. His/her analysis of Blount's data are therefore close to meaningless, regardless of whether the test used by Blount is appropriate or not. In my opinion, the entire page should therefore be deleted. [[User:FredFerguson|FredFerguson]] 08:18, 13 March 2009 (EDT)

::::: "Fred", perhaps you mistakenly think this is Wikipedia, where [[censorship]] and deletion of pages for ideological reasons are common. Not here.--[[User:Aschlafly|Andy Schlafly]] 10:23, 14 March 2009 (EDT)

:::::: Umm... I'm suggesting deletion for mathematical reasons, not ideological reasons. Using an argument filled with mathematical errors to try to support your case only detracts from your credibility. [[User:FredFerguson|FredFerguson]] 10:38, 14 March 2009 (EDT)

:::::: Actually, I think correction is better than deletion. So that's what I've done. [[User:FredFerguson|FredFerguson]] 11:01, 14 March 2009 (EDT)

::::::: I find no credibility in your denial of having ideological reasons.--[[User:Aschlafly|Andy Schlafly]] 11:04, 14 March 2009 (EDT)

== Misinterpretation of test ==

SJohnson, Your analysis misinterprets the test. You say the null hypothesis is that this mutation cannot happen. They saw a mutation (4 mutations, in fact, in the data set you show) so the null hypothesis (as you state is) is disproved. That's perfectly straightforward.

I don't know what the "mean mutation generation" test is but you're doing when you apply a chi-squared test to this dataset is to test if the mutations are evenly distributed throughout the generations. Your test says they are, so there's no strong evidence to suppose that mutations are likely to occur in one generation rather than another in the series of tests. Blount's test says thay aren't, so it's more likely that the mutation will occur later in the series of tests. I can't tell which test is right without knowing more about the test that Blount used.

But that point (the foregoing paragraph) has no bearing at all on the null hypothesis, as you describe it. The mutation appeared, so that means the hypothesis that the mutation can't happen is disproved. Very simple. [[User:FredFerguson|FredFerguson]] 21:10, 8 March 2009 (EDT)

:I never said that “the null hypothesis is that this mutation cannot happen”. The chi-square test statistic I'm using wouldn’t be defined if the null hypothesis mutation rate was zero because the <math>E\left[n_{i,j}\right]</math> term in the denominator of the statistic (see above equation) would be zero.

:The test statistic from the paper is the average of the generation numbers of observed mutations. For experiment one this number is
::<math>
\frac{1}{4}\left(30500+31500+2\times32500\right)= 31750.
</math>
:The same number is shown in Table 2 of the paper. [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

:: SJohnson, the way you're calculating the chi-squared statistic implies that you're testing the null hypothesis of a constant mutation rate over time against an alternative hypothesis of a mutation rate which varies over time. [[User:FredFerguson|FredFerguson]] 11:02, 9 March 2009 (EDT)

As it currently stands, the article makes the following statement: "The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown." This misstates the null hypothesis of the paper, which is elaborated in the Introduction section of the paper, and repeated in the section '''Statistical Analysis of the Replay Experiments''':
<blockquote>
For each experiment, we compared the observed mean generation of those clones that yielded Cit+ variants to the mean expected under the null hypothesis that clones from all generations have equal likelihood. The null thus corresponds to the rare-mutation hypothesis laid out in the Introduction."
Block quote
</blockquote><ref>www.pnas.org/cgi/reprint/105/23/7899.pdf</ref>

The article also continues to describe 'mean mutation generation' as a ''test'' rather than a ''statistic'' to which the ''Monte Carlo test'' was applied.--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

I'm a bit confused about why the Chi-squared test, which we're told compares the results to a null hypothesis of a constant mutation rate, seems insensitive to which generations the Cit+ mutations are found. Instead, the chi-square test seems only to be evaluating whether the frequencies of Cit+ mutations in any particular generation are 'expected'. Thus the test is asking whether finding a distribution (e.g. in the first experiment) across nine periods that have no mutations, two periods that have one mutation and one period with two mutations is a statistically significant deviation from what you'd expect of the mutations were randomly distributed. The number returned from the function is the same regardless of the order of Cit+ results. The number of mutations per bin is not the only question being asked. Instead it's the '''order and temporal distribution''' of Cit+ mutants that the analyses probably need to confront. It's not whether one can get nine no-mutants, two single mutants and one double-mutant result, it's a matter of '''when''' they occur and whether that distribution affects the significance of the results. Blount's hypothesis is that mutations should appear later in the experiment. When formulating a suitable null hypothesis, wouldn't one want to take the timing of Cit+ mutants into consideration too?--[[User:Argon|Argon]] 22:25, 18 March 2009 (EDT)

==Unreferenced Claims==

I deleted the claim that mean mutation generation is an appropriate test statistic because no reference was produced that back that claim. No reference was provided to back the claim that the chi-square test p-values are always conservative, either. [[User:SJohnson|SJohnson]] 12:57, 14 March 2009 (EDT)

: The reference is Everitt. I'll check I put it in the right place. [[User:FredFerguson|FredFerguson]] 13:30, 14 March 2009 (EDT)

There was a typo in my edit summaries on the talk page and the main page. I meant to say "Removed unsupported claims" rather than "Removed supported claims". [[User:SJohnson|SJohnson]] 13:13, 14 March 2009 (EDT)

I do not believe that anyone has claimed that 'chi-square test p-values are ''always'' conservative'. The claim that has been made is that ''under certain circumstances'', namely low n and low individual cell values, the chi-square test is an invalid test; that under those circumstances the power of the test is low and it becomes impossible to reject the null hypothesis even when it is false. You may have missed the pertinent sections in my links above, so I will directly quote the relevant sections. All the quoted sections refer to chi-square testing in particular. Any bolding below is mine.

:This edit claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639379]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

<blockquote>
"Assumptions: 

Even though a nonparametric statistic does not require a normally distributed population, there still are some restrictions regarding its use. 

1. Representative sample (Random) 
2. The data must be in frequency form (nominal data) or greater. 
3. The individual observations must be independent of each other. 
4. '''Sample size must be adequate. In a 2 x 2 table, Chi Square should not be used if n is less than 20. In a larger table, no expected value should be less than 1, and not more than 20% of the variables can have expected values of less than 5'''. 
5. Distribution basis must be decided on before the data is collected. 
6. The sum of the observed frequencies must equal the sum of the expected frequencies."
</blockquote>
<ref>http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm
</ref>

<blockquote>
"Assumptions:
* Random sample data are assumed. As with all significance tests, if you have population data, then any table differences are real and therefore significant. If you have non-random sample data, significance cannot be established, though significance tests are nonetheless sometimes utilized as crude "rules of thumb" anyway.
* A sufficiently large sample size is assumed, as in all significance tests. '''Applying chi-square to small samples exposes the researcher to an unacceptable rate of Type II errors. There is no accepted cutoff. Some set the minimum sample size at 50, while others would allow as few as 20'''. Note chi-square must be calculated on actual count data, not substituting percentages, which would have the effect of pretending the sample size is 100.
* '''Adequate cell sizes are also assumed. Some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count'''. When this assumption is not met, Yates' correction is applied.
* Independence. Observations must be independent. The same observation can only appear in one cell. '''This means chi-square cannot be used to test correlated data (ex., before-after, matched pairs, panel data)'''.
* Similar distribution. Observations must have the same underlying distribution.
* Known distribution. The hypothesized distribution is specified in advance, so that the number of observations that are expected to appear each cell in the table can be calculated without reference to the observed values. Normally this expected value is the crossproduct of the row and column marginals divided by the sample size.
* Non-directional hypotheses are assumed. Chi-square tests the hypothesis that two variables are related only by chance. If a significant relationship is found, this is not equivalent to establishing the researcher's hypothesis that A causes B, or that B causes A.
* Finite values. Observations must be grouped in categories.
* Normal distribution of deviations (observed minus expected values) is assumed. Note chi-square is a nonparametric test in the sense that is does not assume the parameter of normal distribution for the data -- only for the deviations.
* Data level. No assumption is made about level of data. Nominal, ordinal, or interval data may be used with chi-square tests."
</blockquote>
<ref>http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm</ref>

<blockquote>
"Assumptions: 
-None of the expected values may be less than 1 
-No more than 20% of the expected values may be less than 5"</blockquote> <ref>http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html</ref>

<blockquote>
"When performing a chi-square test, your data must satisfy important assumptions. Although these assumptions may be stated differently in different textbooks, they generally assert that: 
1)The sample must be randomly drawn from the population 
'''2)The sample size, n, must be large enough so that the expected cell count in each cell is greater than or equal to 5.''' 
Both assumptions must be met in the process of collecting your data, and violations of the second assumption will appear in the Minitab output when you run the analysis. 
... 
'''You may wonder why the second assumption is necessary for performing the chi-square test. The second assumption arises because the distribution of counts under the null hypothesis is multinomial, and the normal distribution can be used to approximate the multinomial distribution if the sample size is sufficiently large and the probability parameters aren't too small. It can be shown via the Central Limit Theorem that the multinomial distribution converges to the normal distribution as the sample size approaches infinity; however, there is no easy way to show mathematically how and when the convergence fails.'''"
</blockquote>
<ref>http://www.minitab.com/support/docs/Answers/Chi-Square%20Test%20Assumptions.pdf</ref>

<blockquote>
"'''The chi-square test is simpler to calculate but yields only an approximate P value. ... You should definitely avoid the chi-square test when the numbers in the contingency table are very small (any number less than about six)'''."
</blockquote> <ref>http://www.graphpad.com/www/Book/Choose.htm</ref>

<blockquote>
"The most important things to remember to get a valid χ2 test are that the expected values are not too small in any bin (certainly 5 or more), and that the degrees of freedom are properly evaluated. '''Unless you have a very large amount of data, the test is not very sensitive and errs on the side of safety. If you get a significant result, however, it is not likely to be wrong.'''"
</blockquote> <ref>http://mysite.du.edu/~jcalvert/econ/chisquar.htm</ref>

<blockquote>
"The critical assumptions of the chi-square test for k independent samples are similar to those for the chi-square test for two independent samples. ... 
'''4. No more than 20% of the cells may have expected frequencies of less than 5, and no cell should have an expected frequency of less than 1. 
The rule given in Assumption 4 is particularly important for a contingency table that is larger than 2X2'''"
</blockquote> <ref>http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1
</ref>

<blockquote>
"Special problems with small expected cell frequencies for the chi-square test: 
The chi-square test involves using the chi-square distribution to approximate the underlying exact distribution. The approximation becomes better as the expected cell frequencies grow larger, and '''may be inappropriate for tables with very small expected cell frequencies.''' 
'''For tables with expected cell frequencies less than 5, the chi-square approximation may not be reliable. A standard (and conservative) rule of thumb (due to Cochran) is to avoid using the chi-square test for tables with expected cell frequencies less than 1, or when more than 20% of the table cells have expected cell frequencies less than 5.''' 
Another rule of thumb (due to Roscoe and Byars) is that the average expected cell frequency should be at least 1 when the expected cell frequencies are close to equal, and 2 when they are not. (If the chosen significance level is 0.01 instead of 0.05, then double these numbers.) 
Koehler and Larntz suggest that if the total number of observations is at least 10, the number categories is at least 3, and the square of the total number of observations is at least 10 times the number of categories, then the chi-square approximation should be reasonable. 
Care should be taken when cell categories are combined (collapsed together) to fix problems of small expected cell frequencies. Collapsing can destroy evidence of non-independence, so a failure to reject the null hypothesis for the collapsed table does not rule out the possibility of non-independence in the original table. 
'''As with most statistical tests, the power of the chi-square test increases with a larger number of observations. If there are too few observations, it may be impossible to reject the null hypothesis even if it is false.'''"
</blockquote> <ref>http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html</ref>--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

: Thanks for this really excellent contribution, ElyM. The only thing I'd like to add is in relation to your initial statement, "I do not believe that anyone has claimed that 'chi-square test p-values are always conservative'". The question of whether a test is conservative in a particular situation is probabilistic. One can determine whether a test is likely to generate a p-value which is too high in a particular situation (e.g. for a chi-squared test, when there are lots of small expected values) but one needs an exact test (such as an appropriate Monte Carlo randomisation test) to determine whether the p-value in any ''particular'' test is in fact excessively high.

::This edit also claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639373]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

: I hope careful reading of your very clear description will put SJohnson's mind at rest on this subject. [[User:FredFerguson|FredFerguson]] 18:11, 14 March 2009 (EDT)

ElyM, you've provided nothing to address the basic flaw that "The paper incorrectly applied a Monte Carlo resampling test to exclude the null hypothesis for rarely occurring events." See [[Flaws in Lenski Study]]. Also, do not impose your view on the content page until after SJohnson has had an opportunity to respond to your posting. As to "Fred", his put-downs are getting tiresome and I'm going to review his edit pattern now to see if he's been contributing anything of value to this site.--[[User:Aschlafly|Andy Schlafly]] 14:06, 15 March 2009 (EDT)

::Mr. Schlafly, per your request I have not added anything to the content page as SJohnson has not yet responded to my posts. Since all of my comments have been in regards to SJohnson's use of the chi-square test in this particular article, I'm not sure why you expect me to address Blout's use of Monte Carlo - that issue seems to be addressed on the [[Flaws in Lenski Study]] page. SJohnson has added a reformulation of the chi-square test for two possible outcomes, and stated that the chi-square test is at a minimum when all success probabilities are equal. He then extrapolates from this to claim that the chi-square test is an effective test for the data from Blount.

::The reformulation of the equations for two possible outcomes does not address the underlying problem that the chi-square test has universally accepted parameters outside of which it is considered an invalid test; I have provided references for these parameters and shown that the data from Blount lies outside them. None of the expected cells in SJohnson's analysis have values above one, and the total n is four. SJohnson's own reference states that the application of the chi-square test in this circumstance is a "violation of good statistical practice". Analogously, combining F=ma and t=(vf-vi)/a into t=(vf-vi)m/F and showing that t is a minimum when m approaches zero does not address the fact that those Newtonian equations do not apply as velocities approach the speed of light. The legitimacy of Blount's arguments cannot be determined by the application of illegitimate counterarguments. If SJohnson or others can point to references from the statistical literature that show that Blount has made methodological errors - as I have been able to do with SJohnson's chi-square analysis - I would welcome their input, and no doubt Conservapedia's other readers would as well, and this page would be greatly improved.

::I have not seen a rebuttal from SJohnson in the four days since my last post, although he has added new material to the content page since then. In light of this, I would appreciate some guidelines as to when it is appropriate for me to add my information and references to the content page. I can add citations from the primary mathematical literature if necessary, but in general I find that these are less helpful as they are not easily accessible by readers without access to academic libraries.--[[User:ElyM|ElyM]] 18:07, 18 March 2009 (EDT)

::: You say, "I'm not sure why you expect me to address Blout's use of Monte Carlo." The reason is obvious: the title of the content page is the "Significance of E. Coli Evolution Experiments." You haven't addressed the inappropriateness of using Monte Carlo simulations for assessing the significance rarely occurring events, which was central to Lenski's statistical claims. I suggest you address this flaw if you want to be taken seriously.--[[User:Aschlafly|Andy Schlafly]] 23:12, 18 March 2009 (EDT)

::::ASchlafly, again per your request, and based on your statement regarding the "inappropriateness of using Monte Carlo simulations for assessing the significance of rarely occurring events", I have spent the last several days reviewing the literature available to me on Monte Carlo and other resampling techniques, looking for ways in which Blount may have made a methodological error of the sort that SJohnson has made. I have been unable to find any examples of authors suggesting that Monte Carlo be avoided for low ''n'', or for events with low probability regardless of'' n'', much less providing specific cutoff numbers as are seen in the references that I provided for the chi-square test. Similarly, the technique that Blount used does not require/assume that categories are unrelated, as the chi-square test does. Of course, the absence of evidence is not evidence of absence, and I may have misinterpreted the basis of your objection. At this point I'll need you to explain your objection in more detail if you wish me to find the appropriate literature addressing your concerns. Do you believe that the number of resamplings was too low in Blount's paper? That the analysis should have been performed with a software package other than Statistics101? Some other procedural issue? Some issue of interpretation?

::::The statistical problem that Blount must address is straightforward: given a distribution of mutant cultures that ''appears'' to be skewed toward the higher generations, what is the probability that this same amount of skew (or a greater degree) could arise by chance, given the null hypothesis that every generation is equally likely to produce a mutant? Interestingly, in the case of the first replay experiment, the total number of ways to randomly select (equal probability, no replacement) four cultures from seventy-two is 72x71x70x69, or 24,690,960. This number is small enough that a program can brute-force-calculate the 'mean generation number' of ''all possible'' combinations of four cultures in a reasonable amount of time. An experimentally-derived 'mean generation number' can be checked against this exhaustive list, and the number of means equal to or larger than the experimental mean can be found exactly. Converting this number to a percentage of 24,690,960 provides an exact p-value for any given experimental 'mean generation number'. This exhaustive approach is different than the Monte Carlo technique, in that ''all possible'' outcomes are examined, rather than a ''random subset'' of all possible outcomes. For the first replay experiment, it provides a way to independently check Blount's Monte Carlo results. This approach is not possible for the second and third replay experiments, in which the total number of possible combinations becomes impractically large: 340!/335! = 4.41 x10^12 and 2800!/2792! = 3.74 x 10^27, respectively.

::::I asked a colleague to run just such a brute-force program for me on the first replay data. I also ran several Monte Carlo simulations ('''not''' using Statsistics101) with Blount's data, using twenty-five million, one hundred million, and 493,819,200 resamplings - note that this last is twenty times the number of all possible combinations of 4 samples drawn without replacement from 72. The p-values from the 25M, 100M, and 493M Monte Carlo resamplings (0.00844, 0.00846, and 0.00846, respectively) compare favorably with Blount's 1M value of 0.0085 and the non-Monte-Carlo brute-force exact calculation, which provides a p-value of 0.008457. Thus it appears that Blount's statistical results are confirmed by a ''non-Monte Carlo'' technique, at least for the first replay experiment.

:::::What do you get for the experiment two p-value using the method from the paper and at least ten million realizations? [[User:SJohnson|SJohnson]] 08:48, 25 March 2009 (EDT)

::::::For the second replay experiment, Blount reports that one million resamplings gives a p-value of 0.0007. When I run the Monte Carlo simulations, ten million resamplings give a p-value of 0.00060; one hundred million resamplings give a p-value of 0.00062, and one ''billion'' resamplings give a p of 0.00061.

::::::As to the brute-force method for the second replay: the 4.41x10^12 combinations of five cultures picked from 340 actually represents 'only' 36.8 billion unique combinations, since for the purposes of calculating a mean generation value, the ordering of the cultures does not matter: 0, 0, 0, 0, 10 gives the same mean as 10, 0, 0, 0, 0 and 0, 10, 0, 0, 0. With brute force, it turns out that out of the 36,760,655,568 unique combinations possible in the second replay, 22,536,306 have means that are greater than or equal to 32,100.

::::::22,536,306 / 36,760,655,568 = 0.000613 = the ''exact'' p-value derived from exhaustive evaluation rather than Monte Carlo.

::::::The third replay has 9.27 x 10^22 unique combinations; at a billion comparisons a minute it would take over 170,000,000 years to check them all.--[[User:ElyM|ElyM]] 12:36, 25 March 2009 (EDT)

::::::----------
::::::Here are pointers to the freely available Statistics 101 package <ref>http://www.statistics101.net/statistics101web_000003.htm</ref> and the actual programs run through the package by Blount ''et al.'' <ref>http://myxo.css.msu.edu/ecoli/citrate2008/MCprograms.html</ref>. The stats package is written in Java and should run under many operating systems. A 10 million trial run of the second experiment took a bit of time and yielded a p-value of 0.00061. Ten separate, one-million trial runs produced an average p-value of 0.00061 (std.dev=0.00002, n=10). Even with trial sizes of 5K, the numbers averaged about 0.0006 (std.dev=0.0004 n=10).--[[User:Argon|Argon]] 20:52, 25 March 2009 (EDT)

::::My intention is not to get caught up in a digression about Monte Carlo, though - I'd rather keep the focus on the fact that the main article should acknowledge that SJohnson is using chi-square in a way that violates accepted guidelines; this remains true whether Blount's analysis is valid or not.--[[User:ElyM|ElyM]] 12:13, 23 March 2009 (EDT)

== References ==
{{reflist}}

Talk:Significance of E. Coli Evolution Experiments

2009-03-26T00:53:15Z

Argon: Fixed link

SJohnson, your assessment, while good in the utilization of the chi-squared test is unfortunately incorrect. The Monte Carlo resampling gives a more accurate p-value than the chi-squared. You may research the literature (i.e. publications in statistical mathematics, many pubs actualy compare Monte Carlo vs Chi Squared) to discover that this method is commonly used in advance statistical work and how it is more accurate than the chi-squared test.--[[User:Able806|Able806]] 17:00, 4 March 2009 (EST)

:It doesn’t make sense to compare the chi-square test, which is a specific statistical hypothesis test, to Monte Carlo methods, which can be used for anything from fluid motion modeling to p-value computations. You can use Monte Carlo methods to compute the p-values of the chi-square test!

:Monte Carlo methods involve the generation of random realizations. Your broad claim the Monte Carlo methods are “more accurate” than the chi-square test is obviously incorrect because the accuracy of Monte Carlo methods always depends on the number of random realizations generated. When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous.

:Which publications compare Monte Carlo to chi-square and show that the former is more accurate? Could you provide specific examples? Thanks. [[User:SJohnson|SJohnson]] 18:50, 4 March 2009 (EST)

:In furtherance of SJohnson's remarks with respect to rarely occurring events, the use of the basic Monte Carlo method is plainly incorrect for modeling a rarely occurring event, as the Lenski paper did. This has long been pointed out in [[Flaws in Richard Lenski Study]]. I know [[evolutionists]] will never admit a flaw in anything promoting their pet theory, but this (and other) flaws in that paper is undeniable.

:Watch how evolutionists defended obvious errors in the Lenski paper, and then realize why the [[Piltdown Man]] fraud was taught for 40 years without evolutionists admitting it was a hoax.--[[User:Aschlafly|Andy Schlafly]] 09:55, 5 March 2009 (EST)

::Andy, how exactly is the Monte Carlo method incorrect to use in this case? I have seen it used in publications with much smaller datasets.--[[User:Able806|Able806]] 10:29, 5 March 2009 (EST)

:::Able806, I'm interested in looking at the publications you mentioned that use Monte Carlo methods to analyze small data sets. Could you provide some examples? Thanks. [[User:SJohnson|SJohnson]] 16:41, 5 March 2009 (EST)

::::SJohnson, here are two papers, [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WH8-45RFJ1J-19&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=1ad95954654bb97b17e474ce6b469f6e 1] and [http://cat.inist.fr/?aModele=afficheN&cpsidt=787963 2]. Most are in chemistry and genetics where you find the observed to be much smaller and have to use the MCM. You can search on the subject as well and find that how Lenski performed the test is the standard for microbiological genetic analysis.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

:::::Those papers have nothing to do with hypothesis testing. One is an archeology paper. To be blunt, it seems like you’re just doing internet searches on “Monte Carlo” to find these links. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

::::::SJohnson, actually they do, did you read the papers? If so you would see how they used the MCM for their data analysis of small data sets, which indeed was hypothesis testing and answers you inquiry about publications that use MCM for small data set analysis. If you wish I can try to track down some actual mathematical publications, however, I am not as familiar with mathematical journals as I am with science/medical journals (not knowing which mathematical journals are acceptable). I am assuming that you have a background in math and possibly access to mathematical journals, therefore if you know the reputable ones I can do the leg work.
::::::I believe the thing that needs to be looked at is there truly a problem with the choice of test and if so what is an alternative. Bayesian might be an option but seems to be difficult to employ for this situation.--[[User:Able806|Able806]] 12:36, 12 March 2009 (EDT)

:::Able806, you still seem to miss the point about how inappropriate the Monte Carlo method (as used in the Lenski paper) is for evaluating rarely occurring events. You need to open your mind to be productive. If you simply cling to a view that Lenski (who I don't think has any meaningful education in statistics) must somehow be right, then you're not going to make any progress in understanding the flaws.--[[User:Aschlafly|Andy Schlafly]] 17:07, 5 March 2009 (EST)

::::Andy, you still have not answered what you find inappropriate about his use of the Monte Carlo method? I am a reasonable person and with evidence I do have an open mind. I provided examples last week, with a working model, showing that Monte Carlo is better than the chi-square in this case. I have also shown where the Chi-Square was inappropriate due to the occurrence size as well. So if you have any evidence that Monte Carlo should not be used in the way that Lenski used please let it be shown.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

Sjohnson, I believe you just proved my point. In the literature of mean and covariance structure analysis, non-central chi-square distribution is commonly used to describe the behavior of the likelihood ratio statistic under alternative hypothesis; it is widely believed that the non-central chi-square distribution is justified by statistical theory. Actually, when the null hypothesis is not trivially violated, the non-central chi-square distribution cannot describe the LR statistic well even when data are normally distributed and the sample size is large. Monte Carlo results compare the strength of the normal distribution against that of the non-central chi-square distribution. In an association analysis comparing cases and controls with respect to allele frequencies at a highly polymorphic locus, a potential problem is that the conventional chi-squared test may not be valid for a large, sparse contingency table. Reliance on statistics with known asymptotic distribution is unnecessary, as Monte Carlo simulations can be performed to estimate the significance level of the test statistic.

Here is a [http://faculty.vassar.edu/lowry/chi_beta.html link] to a great page the provides an interactive example as to why the Chi Squared test would provide poor results compared to the Monte Carlo in relation to the Lenski data workup.

Something you may have overlooked was that the data set is actually too small to use the chi square method correctly. It is often accepted that is any of the analyzed data falls under 10 for a particular cell of the data set then the Yates correction needs to be applied; unfortunately the Yates correction can over correct thus skewing the p-value. Lenksi seemed to understand this by supporting his Monte Carlo p-value results with the Fisher z-transformation p-value.

I hope this helps.--[[User:Able806|Able806]] 10:27, 5 March 2009 (EST)

:I’m still waiting to hear which literature says that “Monte Carlo resampling” is “more accurate than the chi-squared test”. The page mentioned above [http://faculty.vassar.edu/lowry/chi_beta.html] is a discussion of why statisticians “fail to reject the null” rather than “accepting the null” when the p-value is above 0.05 or so. The page says nothing about superiority of Monte Carlo methods. Why were alternate hypothesis distributions mentioned? Only the null hypothesis distribution is used to calculate a p-value. Yates’s correction is for 2x2 contingency tables [http://en.wikipedia.org/wiki/Yates%27_correction_for_continuity]. It doesn’t apply in this case. Finally, what the heck do “covariance structure analysis” and “allele frequencies at a highly polymorphic locus” have to do with this problem? [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::SJohnson, I am looking for this paper for you, I cited it for one of my past publications dealing with allele frequencies (I believe it came from the Duke Biostatistics group). To answer your question about allele frequencies, that is the issue at hand, more about the genetics than the math, but it is the item being studied. So you stated that Yates can not be used and statistics says the number of occurrences is too small to evaluate using the Chi-Squared test so what would you recommend instead of the Monte-Carlo Method?

:Regarding the "Fisher z-transformation p-value" from the paper, garbage in garbage out. If the p-values were bad to begin with, then why would a combination of them be meaningful? [[User:SJohnson|SJohnson]] 10:49, 9 March 2009 (EDT)

::You are assuming that p-values are wrong based on a test that is inappropriate in this case due to data limitations. Did you perform a z-transformation on the chi-squared for the three data groups?--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

::You asked about the “Fisher z-transformation p-value”. The z-transformation test and Fisher’s method are actually two different things (see Whitlock's 2005 paper - Ref. 49 in Blount et al.). But no, I haven’t tried either. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::There's a large literature on various kinds of Monte Carlo test, a very short summary of which is that they're inevitably more accurate than parametric tests (e.g. F, t, chi-squared, etc) because they don't make assumptions about the distribution of the data under the null hypothesis. See for example ''Introduction to the Bootstrap'' by B. Efron and R. Tibshirani and ''The Jack-knife, the Bootstrap and Other Resampling Plans'', also by Efron. They're certainly applicable to small datasets and their accuracy is really only limited by the number of samples you care to take. E.g. 1000 M-C samples would give you a pretty accurate idea about significance at the alpha<1% level (That book should answer SJohnson's questions of 18:50 on 4/3/09 and 16:38 on 5/3/09 about accuracy and Aschalfly's comment of 17:07 on 5/3/09 about appropriateness of Monte Carlo tests.) [[User:FredFerguson|FredFerguson]] 16:53, 11 March 2009 (EDT)

::::Your claim that Monte Carlo methods are “inevitably more accurate” than other tests is obviously wrong because the accuracy of MC methods always depends on the number of realizations used. You should have written <math>\alpha=1\%</math>, not <math>\alpha<1\%</math>. If 1,000 random realizations are generated, the number of realizations above the true <math>\alpha=1\%</math> level is binomial with mean 10 and variance about 10. Thus, the standard deviation of the MC estimate is >0.003. In this example, a Monte Carlo p-value could be off by 30% and still be within a standard deviation. Is that really “pretty accurate”?

::::Using one million MC realizations (as done in the paper) at the <math>\alpha=0.001</math> level means the standard deviation is about 3%. The paper reported a p-value of less than 0.001 (experiment two). It wouldn’t surprise me to find out that the experiment two p-value for the flawed test is off because only one million realizations were used. My original statement, “When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous” is correct. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::::You're talking about miniscule differences in the accuracy of a test. 0.013 isn't very different from 0.007. In either case, it's very unlikely the experimenter would have obtained that result if the null hypothesis were true. If you're bothered about differences in P-values to the third decimals (which would make you unusual!), just run more MC realisations, that's all. Not really a problem. [[User:FredFerguson|FredFerguson]] 11:53, 12 March 2009 (EDT)

There’s still confusion about the difference between test statistics and Monte Carlo methods. Before you find a Monte Carlo estimate of a p-value, you need to select a test statistic to reduce the data set to a scalar. I am interested in hearing which test statistic you believe should be used in place of the chi-square test and why. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

----

Quick question for SJohnson: How many degrees of freedom did you choose when calculating the p-value? I'd like to know upon what condition you base that number. Thanks.--[[User:Argon|Argon]] 11:05, 5 March 2009 (EST)

:The degree of freedom for a contingency table is rows minus one times columns minus one. That is, <math> (r-1)(c-1) </math>. Here’s a pretty good tutorial I came across: [http://faculty.uncfsu.edu/dwallace/lesson%2020.pdf]. For the experiments from [http://www.pnas.org/content/105/23/7899.full.pdf], the DOFs are 11, 11, and 13. For experiment one, the chi-square test statistic is
::<math>
X^2
=\sum\limits_i\sum\limits_j
\frac{\left(n_{i,j}-E\left[n_{i,j}\right]\right)^2}
{E\left[n_{i,j}\right]}
</math>
::<math>
=\frac{\left(0-1/3\right)^2}{1/3}
+\frac{\left(6-17/3\right)^2}{17/3}
+\frac{\left(0-1/3\right)^2}{1/3}
+\ldots+
</math>
::<math>
+\frac{\left(2-1/3\right)^2}{1/3}
+\frac{\left(4-17/3\right)^2}{17/3}
</math>
::<math>
\approx
14.82
</math>
:where <math>n_{i,j}</math> is the observed value and <math>E\left[n_{i,j}\right]</math> is the expected null hypothesis value. So if you have MS Excel, another way to arrive at the p-value of 0.19 is to type “=CHIDIST(14.82,11)” into a cell. Cheers! [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::OK, thanks for the info. From what I'd calculated and looked up in tables, the numbers seemed close to a df=11 for a chi-square of ~14. (Aside: With terms having 17/3 in the denominator in the figures above, were you using the test of independence? I was using Pearson's test for [http://en.wikipedia.org/wiki/Pearson%27s_chi-square_test#Test_for_fit_of_a_distribution fit of a distribution] which returns a chi-squared value of 14 and roughly matched the p-values you reported, assuming the df was 11).

::Also, the first sentence of the article reads: "Blount, Borland, and Lenski[1] claimed that a key evolutionary innovation was observed during a laboratory experiment. That claim is false." A small correction: There were several claims in the paper. The 'key evolutionary innovation' was acquiring the ability to utilize citrate as a food source. That claim was demonstrated multiple times. The claim, which pertains to this statistics discussion was that the Cit+ phenotype arose in a multi-step process, first requiring a rare, pre-adaptive mutation before additional mutation(s) lead to the subsequent development of citrate utilization.--[[User:Argon|Argon]] 20:46, 5 March 2009 (EST)

:::My biology-degreed wife assures me that mutation does not necessarily mean that evolution occurred. What the paper claimed is that evolution (a “key innovation”) occurred in the lab. The key innovation supposedly increased the mutation rate. In the experiments, the observed mutation rate increased after generation 31,000, but not enough to make a statistically significant claim that the rate is not constant. The analysis in the paper was similar to flipping a coin ten times, counting six heads and claiming that the coin must be biased against tails. In reality, there’s nothing surprising about a fair coin producing slightly more of one outcome than the other. Just like there's nothing surprising about there being slightly more mutations in later generations than early generations given the null hypothesis (constant mutation rate). [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

>>Inserting a later comment first<<
SJohnson, the paper's title is: "Historical contingency '''and the evolution of a key innovation''' in an experimental population of ''Escherichia coli''" As I mentioned earlier, the key innovation is the evolution of the Cit+ phenotype and not the timing or rate of its acquisition. And yes, it *is* evolution (call it microevolution, if you wish). Blount et al went on further to speculate how this evolutionary innovation arose and they proposed the historical contingency hypothesis in which 'pre-adaptive' mutations were required before the Cit+ phenotype developed. It is only this latter hypothesis that you are attempting to address with your chi-square analysis, not the fact that Cit+ mutants arose (which is the evolutionary innovation).--[[User:Argon|Argon]] 21:57, 18 March 2009 (EDT)

::::SJohnson, not to say anything about your wife, but has she had a 400 level molecular genetics course (most general biology degrees do not cover the detail unless they are specialized)? If so, she would have mentioned that if the mutation passes to the offspring and is selectively beneficial to the population then it is a step of evolution as along as the conditions continue through the sharing of the mutation with the population and the environment is such that reduces the growth rate of the non-transformed population. While not all mutations are signs that evolution occurred the mutations that pass to offspring and provide a benefit compared to other offspring are very strong indicators. In the case of this paper the population that evolved the cit+ was able to metabolize a chemical in their environment which allowed for an adaptation advantage compared to the non-transformed colonies.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

----

Let’s go back to the beginning. There appears to be confusion about the difference between test statistics and methods for computing p-values. As is noted at the beginning of the page [http://www.conservapedia.com/Significance_of_E._Coli_Evolution_Experiments], the fundamental problem with the paper is that it used a flawed test statistic, not that it used Monte Carlo methods to find the p-value for that flawed statistic.

Every hypothesis test uses a test statistic to reduce the data to a single number. The p-value for the test statistic can be calculated analytically (as I’ve done for the chi-square test statistic) or by Monte Carlo methods. In the paper, Monte Carlo methods were used to compute the p-value of the “mutation generation” test statistic. The key problem with the analysis from the paper is that it doesn’t work to use a weighted average to test for variations in mutation rate. This is like trying to use the sample variance to test for an increase in the mean in Gaussian-distributed data. A statistic should be selected based on the null and alternate hypothesis distributions of the data. The chi-square test (unlike the weighted average from the paper) is a reasonable choice for data that mutates at a constant rate under the null hypothesis, but mutates at varying rates under the alternate hypothesis.

Able806, you made a good point about the contingency table cell frequencies being relatively low, but were wrong when you said ”the data set is actually too small to use the chi square method correctly”. In the low cell frequency case the chi-square test is still effective, but the null hypothesis distribution of the chi-square statistic starts to look less like the chi-square distribution. Thus, p-values calculated using the chi-square distribution may be a bit off. However, Monte Carlo p-values are always imperfect as well because it's impossible to generate an infinite number of random realizations. There are imperfections in p-values generated by analytic and Monte Carlo methods. However, low cell frequencies does not explain the >20x and >2.5x differences between chi-square p-values and p-values from the paper for experiments one and three. The reason for those huge differences was the use of the flawed test statistic (“mutation generation”) in the paper. [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

:SJohnson, the chi-squared test is a valuable statistical tool, but the limitations of the test must be acknowledged. The chi-squared test can only produce valid results if the assumptions that underly the test are not violated. As an analogy, Newtonian models of motion fail to produce accurate results as velocities approach the speed of light; under those circumstances one must switch to a theory that accounts for relativistic effects.

:It seems that you have simply dismissed the [http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm widely-acknowledged] [http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm fact] that the [http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html chi-squared test] is [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236 inappropriate] for use in [http://www.graphpad.com/www/Book/Choose.htm situations] where n in any cell is [http://mysite.du.edu/~jcalvert/econ/chisquar.htm less] less than a [http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1 threshold] [http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html number]. Different authors set different thresholds, but all are well above the numbers seen in your chi-squared analysis - even the most liberal guidelines advise against the chi-squared test when any expected cell frequency is less than one or more than 20% of the table cells are less than 5; others require that expected values in all cells must be more than 5. With smaller amounts of data, the test is insensitive and errs on the side of rejecting the hypothesis. If you attempt your chi-squared statistical analysis with a program that is more sophisticated than MS Excel (as I did), you get an error message indicating that the results are invalid due to low expected cell counts.

:That issue aside, there are other reasons that the chi-squared test is inappropriate here. As the links above point out, the categories tested must be truly independent; one example is that you can't use the chi-squared test to compare age and ability to kick a field goal by testing the same experimental group twice, one year apart; you have to test one group of age A and a different group of age B. In the case of the Blount paper, the categories are not independent. Even if there were adequate numbers to address the low-expected-frequency problem, this would make the chi-squared an invalid test in this case.

:There are other significant problems with the use of the chi-squared test in this circumstance, but they can wait until you address these first major problems.--[[User:ElyM|ElyM]] 12:18, 11 March 2009 (EDT)

::Wackerly et al. says in general it’s assumed that the cell frequencies are above five so that the chi-square statistic (under the null) is approximately chi-square distributed (see p. 703). That book does not say chi-square test results are invalid if frequencies are five or less. Your example of a chi-square test warning message (it said "warning" not "error" as you stated) in Minitab [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236] said “approximation probably invalid” referring to the chi-square distribution approximation to the chi-square test statistic’s distribution. Your example did not say “chi-square test invalid”. I agree that when cell frequencies are low, the chi-square test statistic’s distribution starts to deviate from the chi-square distribution. I maintain that this deviation is not enough to explain the >2.5x and >20x differences in the chi-square test p-values and the p-values from the paper.

::As the numerous links in your post proved, the chi-square test is widely-used by statisticians. Can you give examples of statisticians using mean mutation generation as a test statistic? Also, did your software agree with the chi-square test p-values I presented? Thanks. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::Thank you for giving page references for Wackerly; however it seems we have different editions, since page 703 in my copy (5th ed, 1996) does not deal with chi-squared issues at all. My copy does state the following, on page 622: "Although the mathematical proof is beyond the scope of this text, it can be shown that, when n is large [chi-squared] will possess approximately a chi-square probability distribution in repeated sampling." Then, on page 624: "Experience has shown that cell counts [n sub i] should not be too small in order that the chi-square distribution provide an accurate approximation to the distribution of [chi squared]. As a rule of thumb we require that all expected cell counts equal or exceed 5, although Cochran (1952) has noted that this value can be as low as 1 for some situations." Wackerly then goes on, in the problems sections, to describe the use of the chi-squared test as a "violation of good statistical practice" when "some expected counts [are] <5."

:::It seems that you are already aware that the [chi-square] statistic under the null is no longer chi-square distributed for small n; this is precisely why the test should not be used under those conditions. I can claim to be able to accelerate a 1-kg mass to 10 times the speed of light by applying 1 N of force for 95 years by using F=ma and t= (vf-vi)/a. Plugging the numbers into those equations will produce the same result every time, but the answer is illegitimate because those equations are only valid under certain assumptions, which are violated as velocities approach the speed of light. Similarly, having a statistical program calculate a chi-squared value given the Blount data will produce a number result, but since the assumptions of the test are violated the result is not legitimate. Yes, if I put the Blount data in SAS 9.2, I get the same numerical answer as you do, but I also get the following message: "WARNING: >89% of the cells have expected counts less than 5. Chi-square may not be a valid test." You may argue that that's a warning, not an error; that's a semantic distinction. The reason that the program says that it MAY not be valid is that the chi-squared test skews in the direction of being too conservative at low n values; the test has an acceptable rate of false positives but an unacceptably high rate of false negatives. Comparing the results of the Monte Carlo and chi-squared results in this case is like comparing the results of Newtonian and relativistic equations of motion: they can produce very different results from the same input data.

::::For a finite amount of data, the chi-square statistic is never chi-square distributed under the null. The p-values are always approximate regardless of cell frequencies. The approximation becomes more accurate as the amount of data increases, but I don’t believe that this inaccuracy will change p-values that are about 0.2 (for experiments 1 and 3) into statistically significant p-values. How much do you expect the p-values to change if an exact computation is used in place of the chi-square distribution approximation? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::Your last paragraph has a major non sequitur in it: yes, many statisticians use the chi-square test. As long as the assumptions of the test are not violated, it is a valuable tool. That has nothing to do with the validity of using mean mutation generation as a test statistic. 'Mean number of werewolf attacks in Mumbai in the week centered on the new moon, by month, from 1654 to 1798' is a valid test statistic. I am quite sure that it has never been used in a peer-reviewed paper before. That does not mean that I can't perform valid statistical tests on that statistic. If, however, the incorrect test is applied, the results of the analysis will be flawed. Papers apply a (relatively small) standard repertoire of valid tests to a (potentially infinite) number of test statistics. The particular test statistic used in a paper may never have been used before and may never be used again; that does not address the validity of the analysis. In Blount's case, the test is the Monte Carlo analysis, which is also "widely-used by statisticians".

::::There are an infinite number of ways to reduce a data set to a single number. However, it’s foolish to think every method would be effective. I gave an example of a flawed test statistic in an earlier post [http://www.conservapedia.com/index.php?title=Talk%3ASignificance_of_E._Coli_Evolution_Experiments&diff=635070&oldid=634987]. Another example of a flawed test statistic is the one used in the paper because it does not always detect deviations from the null hypothesis (see: [[Significance of E. Coli Evolution Experiments#Test Statistics]]).

::::Test statistics are typically derived. The likelihood ratio test is a common method used to derive them. The chi-square test for independence is an approximation to the LRT. Where is the derivation saying that mean mutation generation is an appropriate test statistic for this problem? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::We still haven't touched on the issue of the categories not being independent, which by itself is sufficient to invalidate the chi-squared technique. I'm new to this site, so I'm unsure as to the etiquette of making changes to the articles of another person - but the article here should at the very least mention that the chi-square test is being used here in a manner that violates its underlying assumptions in at least two fundamental ways, and the results are therefore suspect.--[[User:ElyM|ElyM]] 17:34, 12 March 2009 (EDT)

:::::When generating random realizations of experiment outcomes, the authors assumed that the total number of mutants was fixed. Thus the paper assumed the numbers of mutants per generation are statistically dependent. Does this seem like a realistic model, or do you think that if the experiments were recreated that the total number of mutants could vary? For example, if experiment one were recreated, would the total number of mutants always be exactly four? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::: It looks to me as though SJohnson has misinterpreted the application of the chi-squared test in quite a fundamental way. His/her analysis of Blount's data are therefore close to meaningless, regardless of whether the test used by Blount is appropriate or not. In my opinion, the entire page should therefore be deleted. [[User:FredFerguson|FredFerguson]] 08:18, 13 March 2009 (EDT)

::::: "Fred", perhaps you mistakenly think this is Wikipedia, where [[censorship]] and deletion of pages for ideological reasons are common. Not here.--[[User:Aschlafly|Andy Schlafly]] 10:23, 14 March 2009 (EDT)

:::::: Umm... I'm suggesting deletion for mathematical reasons, not ideological reasons. Using an argument filled with mathematical errors to try to support your case only detracts from your credibility. [[User:FredFerguson|FredFerguson]] 10:38, 14 March 2009 (EDT)

:::::: Actually, I think correction is better than deletion. So that's what I've done. [[User:FredFerguson|FredFerguson]] 11:01, 14 March 2009 (EDT)

::::::: I find no credibility in your denial of having ideological reasons.--[[User:Aschlafly|Andy Schlafly]] 11:04, 14 March 2009 (EDT)

== Misinterpretation of test ==

SJohnson, Your analysis misinterprets the test. You say the null hypothesis is that this mutation cannot happen. They saw a mutation (4 mutations, in fact, in the data set you show) so the null hypothesis (as you state is) is disproved. That's perfectly straightforward.

I don't know what the "mean mutation generation" test is but you're doing when you apply a chi-squared test to this dataset is to test if the mutations are evenly distributed throughout the generations. Your test says they are, so there's no strong evidence to suppose that mutations are likely to occur in one generation rather than another in the series of tests. Blount's test says thay aren't, so it's more likely that the mutation will occur later in the series of tests. I can't tell which test is right without knowing more about the test that Blount used.

But that point (the foregoing paragraph) has no bearing at all on the null hypothesis, as you describe it. The mutation appeared, so that means the hypothesis that the mutation can't happen is disproved. Very simple. [[User:FredFerguson|FredFerguson]] 21:10, 8 March 2009 (EDT)

:I never said that “the null hypothesis is that this mutation cannot happen”. The chi-square test statistic I'm using wouldn’t be defined if the null hypothesis mutation rate was zero because the <math>E\left[n_{i,j}\right]</math> term in the denominator of the statistic (see above equation) would be zero.

:The test statistic from the paper is the average of the generation numbers of observed mutations. For experiment one this number is
::<math>
\frac{1}{4}\left(30500+31500+2\times32500\right)= 31750.
</math>
:The same number is shown in Table 2 of the paper. [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

:: SJohnson, the way you're calculating the chi-squared statistic implies that you're testing the null hypothesis of a constant mutation rate over time against an alternative hypothesis of a mutation rate which varies over time. [[User:FredFerguson|FredFerguson]] 11:02, 9 March 2009 (EDT)

As it currently stands, the article makes the following statement: "The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown." This misstates the null hypothesis of the paper, which is elaborated in the Introduction section of the paper, and repeated in the section '''Statistical Analysis of the Replay Experiments''':
<blockquote>
For each experiment, we compared the observed mean generation of those clones that yielded Cit+ variants to the mean expected under the null hypothesis that clones from all generations have equal likelihood. The null thus corresponds to the rare-mutation hypothesis laid out in the Introduction."
Block quote
</blockquote><ref>www.pnas.org/cgi/reprint/105/23/7899.pdf</ref>

The article also continues to describe 'mean mutation generation' as a ''test'' rather than a ''statistic'' to which the ''Monte Carlo test'' was applied.--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

I'm a bit confused about why the Chi-squared test, which we're told compares the results to a null hypothesis of a constant mutation rate, seems insensitive to which generations the Cit+ mutations are found. Instead, the chi-square test seems only to be evaluating whether the frequencies of Cit+ mutations in any particular generation are 'expected'. Thus the test is asking whether finding a distribution (e.g. in the first experiment) across nine periods that have no mutations, two periods that have one mutation and one period with two mutations is a statistically significant deviation from what you'd expect of the mutations were randomly distributed. The number returned from the function is the same regardless of the order of Cit+ results. The number of mutations per bin is not the only question being asked. Instead it's the '''order and temporal distribution''' of Cit+ mutants that the analyses probably need to confront. It's not whether one can get nine no-mutants, two single mutants and one double-mutant result, it's a matter of '''when''' they occur and whether that distribution affects the significance of the results. Blount's hypothesis is that mutations should appear later in the experiment. When formulating a suitable null hypothesis, wouldn't one want to take the timing of Cit+ mutants into consideration too?--[[User:Argon|Argon]] 22:25, 18 March 2009 (EDT)

==Unreferenced Claims==

I deleted the claim that mean mutation generation is an appropriate test statistic because no reference was produced that back that claim. No reference was provided to back the claim that the chi-square test p-values are always conservative, either. [[User:SJohnson|SJohnson]] 12:57, 14 March 2009 (EDT)

: The reference is Everitt. I'll check I put it in the right place. [[User:FredFerguson|FredFerguson]] 13:30, 14 March 2009 (EDT)

There was a typo in my edit summaries on the talk page and the main page. I meant to say "Removed unsupported claims" rather than "Removed supported claims". [[User:SJohnson|SJohnson]] 13:13, 14 March 2009 (EDT)

I do not believe that anyone has claimed that 'chi-square test p-values are ''always'' conservative'. The claim that has been made is that ''under certain circumstances'', namely low n and low individual cell values, the chi-square test is an invalid test; that under those circumstances the power of the test is low and it becomes impossible to reject the null hypothesis even when it is false. You may have missed the pertinent sections in my links above, so I will directly quote the relevant sections. All the quoted sections refer to chi-square testing in particular. Any bolding below is mine.

:This edit claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639379]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

<blockquote>
"Assumptions: 

Even though a nonparametric statistic does not require a normally distributed population, there still are some restrictions regarding its use. 

1. Representative sample (Random) 
2. The data must be in frequency form (nominal data) or greater. 
3. The individual observations must be independent of each other. 
4. '''Sample size must be adequate. In a 2 x 2 table, Chi Square should not be used if n is less than 20. In a larger table, no expected value should be less than 1, and not more than 20% of the variables can have expected values of less than 5'''. 
5. Distribution basis must be decided on before the data is collected. 
6. The sum of the observed frequencies must equal the sum of the expected frequencies."
</blockquote>
<ref>http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm
</ref>

<blockquote>
"Assumptions:
* Random sample data are assumed. As with all significance tests, if you have population data, then any table differences are real and therefore significant. If you have non-random sample data, significance cannot be established, though significance tests are nonetheless sometimes utilized as crude "rules of thumb" anyway.
* A sufficiently large sample size is assumed, as in all significance tests. '''Applying chi-square to small samples exposes the researcher to an unacceptable rate of Type II errors. There is no accepted cutoff. Some set the minimum sample size at 50, while others would allow as few as 20'''. Note chi-square must be calculated on actual count data, not substituting percentages, which would have the effect of pretending the sample size is 100.
* '''Adequate cell sizes are also assumed. Some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count'''. When this assumption is not met, Yates' correction is applied.
* Independence. Observations must be independent. The same observation can only appear in one cell. '''This means chi-square cannot be used to test correlated data (ex., before-after, matched pairs, panel data)'''.
* Similar distribution. Observations must have the same underlying distribution.
* Known distribution. The hypothesized distribution is specified in advance, so that the number of observations that are expected to appear each cell in the table can be calculated without reference to the observed values. Normally this expected value is the crossproduct of the row and column marginals divided by the sample size.
* Non-directional hypotheses are assumed. Chi-square tests the hypothesis that two variables are related only by chance. If a significant relationship is found, this is not equivalent to establishing the researcher's hypothesis that A causes B, or that B causes A.
* Finite values. Observations must be grouped in categories.
* Normal distribution of deviations (observed minus expected values) is assumed. Note chi-square is a nonparametric test in the sense that is does not assume the parameter of normal distribution for the data -- only for the deviations.
* Data level. No assumption is made about level of data. Nominal, ordinal, or interval data may be used with chi-square tests."
</blockquote>
<ref>http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm</ref>

<blockquote>
"Assumptions: 
-None of the expected values may be less than 1 
-No more than 20% of the expected values may be less than 5"</blockquote> <ref>http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html</ref>

<blockquote>
"When performing a chi-square test, your data must satisfy important assumptions. Although these assumptions may be stated differently in different textbooks, they generally assert that: 
1)The sample must be randomly drawn from the population 
'''2)The sample size, n, must be large enough so that the expected cell count in each cell is greater than or equal to 5.''' 
Both assumptions must be met in the process of collecting your data, and violations of the second assumption will appear in the Minitab output when you run the analysis. 
... 
'''You may wonder why the second assumption is necessary for performing the chi-square test. The second assumption arises because the distribution of counts under the null hypothesis is multinomial, and the normal distribution can be used to approximate the multinomial distribution if the sample size is sufficiently large and the probability parameters aren't too small. It can be shown via the Central Limit Theorem that the multinomial distribution converges to the normal distribution as the sample size approaches infinity; however, there is no easy way to show mathematically how and when the convergence fails.'''"
</blockquote>
<ref>http://www.minitab.com/support/docs/Answers/Chi-Square%20Test%20Assumptions.pdf</ref>

<blockquote>
"'''The chi-square test is simpler to calculate but yields only an approximate P value. ... You should definitely avoid the chi-square test when the numbers in the contingency table are very small (any number less than about six)'''."
</blockquote> <ref>http://www.graphpad.com/www/Book/Choose.htm</ref>

<blockquote>
"The most important things to remember to get a valid χ2 test are that the expected values are not too small in any bin (certainly 5 or more), and that the degrees of freedom are properly evaluated. '''Unless you have a very large amount of data, the test is not very sensitive and errs on the side of safety. If you get a significant result, however, it is not likely to be wrong.'''"
</blockquote> <ref>http://mysite.du.edu/~jcalvert/econ/chisquar.htm</ref>

<blockquote>
"The critical assumptions of the chi-square test for k independent samples are similar to those for the chi-square test for two independent samples. ... 
'''4. No more than 20% of the cells may have expected frequencies of less than 5, and no cell should have an expected frequency of less than 1. 
The rule given in Assumption 4 is particularly important for a contingency table that is larger than 2X2'''"
</blockquote> <ref>http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1
</ref>

<blockquote>
"Special problems with small expected cell frequencies for the chi-square test: 
The chi-square test involves using the chi-square distribution to approximate the underlying exact distribution. The approximation becomes better as the expected cell frequencies grow larger, and '''may be inappropriate for tables with very small expected cell frequencies.''' 
'''For tables with expected cell frequencies less than 5, the chi-square approximation may not be reliable. A standard (and conservative) rule of thumb (due to Cochran) is to avoid using the chi-square test for tables with expected cell frequencies less than 1, or when more than 20% of the table cells have expected cell frequencies less than 5.''' 
Another rule of thumb (due to Roscoe and Byars) is that the average expected cell frequency should be at least 1 when the expected cell frequencies are close to equal, and 2 when they are not. (If the chosen significance level is 0.01 instead of 0.05, then double these numbers.) 
Koehler and Larntz suggest that if the total number of observations is at least 10, the number categories is at least 3, and the square of the total number of observations is at least 10 times the number of categories, then the chi-square approximation should be reasonable. 
Care should be taken when cell categories are combined (collapsed together) to fix problems of small expected cell frequencies. Collapsing can destroy evidence of non-independence, so a failure to reject the null hypothesis for the collapsed table does not rule out the possibility of non-independence in the original table. 
'''As with most statistical tests, the power of the chi-square test increases with a larger number of observations. If there are too few observations, it may be impossible to reject the null hypothesis even if it is false.'''"
</blockquote> <ref>http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html</ref>--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

: Thanks for this really excellent contribution, ElyM. The only thing I'd like to add is in relation to your initial statement, "I do not believe that anyone has claimed that 'chi-square test p-values are always conservative'". The question of whether a test is conservative in a particular situation is probabilistic. One can determine whether a test is likely to generate a p-value which is too high in a particular situation (e.g. for a chi-squared test, when there are lots of small expected values) but one needs an exact test (such as an appropriate Monte Carlo randomisation test) to determine whether the p-value in any ''particular'' test is in fact excessively high.

::This edit also claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639373]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

: I hope careful reading of your very clear description will put SJohnson's mind at rest on this subject. [[User:FredFerguson|FredFerguson]] 18:11, 14 March 2009 (EDT)

ElyM, you've provided nothing to address the basic flaw that "The paper incorrectly applied a Monte Carlo resampling test to exclude the null hypothesis for rarely occurring events." See [[Flaws in Lenski Study]]. Also, do not impose your view on the content page until after SJohnson has had an opportunity to respond to your posting. As to "Fred", his put-downs are getting tiresome and I'm going to review his edit pattern now to see if he's been contributing anything of value to this site.--[[User:Aschlafly|Andy Schlafly]] 14:06, 15 March 2009 (EDT)

::Mr. Schlafly, per your request I have not added anything to the content page as SJohnson has not yet responded to my posts. Since all of my comments have been in regards to SJohnson's use of the chi-square test in this particular article, I'm not sure why you expect me to address Blout's use of Monte Carlo - that issue seems to be addressed on the [[Flaws in Lenski Study]] page. SJohnson has added a reformulation of the chi-square test for two possible outcomes, and stated that the chi-square test is at a minimum when all success probabilities are equal. He then extrapolates from this to claim that the chi-square test is an effective test for the data from Blount.

::The reformulation of the equations for two possible outcomes does not address the underlying problem that the chi-square test has universally accepted parameters outside of which it is considered an invalid test; I have provided references for these parameters and shown that the data from Blount lies outside them. None of the expected cells in SJohnson's analysis have values above one, and the total n is four. SJohnson's own reference states that the application of the chi-square test in this circumstance is a "violation of good statistical practice". Analogously, combining F=ma and t=(vf-vi)/a into t=(vf-vi)m/F and showing that t is a minimum when m approaches zero does not address the fact that those Newtonian equations do not apply as velocities approach the speed of light. The legitimacy of Blount's arguments cannot be determined by the application of illegitimate counterarguments. If SJohnson or others can point to references from the statistical literature that show that Blount has made methodological errors - as I have been able to do with SJohnson's chi-square analysis - I would welcome their input, and no doubt Conservapedia's other readers would as well, and this page would be greatly improved.

::I have not seen a rebuttal from SJohnson in the four days since my last post, although he has added new material to the content page since then. In light of this, I would appreciate some guidelines as to when it is appropriate for me to add my information and references to the content page. I can add citations from the primary mathematical literature if necessary, but in general I find that these are less helpful as they are not easily accessible by readers without access to academic libraries.--[[User:ElyM|ElyM]] 18:07, 18 March 2009 (EDT)

::: You say, "I'm not sure why you expect me to address Blout's use of Monte Carlo." The reason is obvious: the title of the content page is the "Significance of E. Coli Evolution Experiments." You haven't addressed the inappropriateness of using Monte Carlo simulations for assessing the significance rarely occurring events, which was central to Lenski's statistical claims. I suggest you address this flaw if you want to be taken seriously.--[[User:Aschlafly|Andy Schlafly]] 23:12, 18 March 2009 (EDT)

::::ASchlafly, again per your request, and based on your statement regarding the "inappropriateness of using Monte Carlo simulations for assessing the significance of rarely occurring events", I have spent the last several days reviewing the literature available to me on Monte Carlo and other resampling techniques, looking for ways in which Blount may have made a methodological error of the sort that SJohnson has made. I have been unable to find any examples of authors suggesting that Monte Carlo be avoided for low ''n'', or for events with low probability regardless of'' n'', much less providing specific cutoff numbers as are seen in the references that I provided for the chi-square test. Similarly, the technique that Blount used does not require/assume that categories are unrelated, as the chi-square test does. Of course, the absence of evidence is not evidence of absence, and I may have misinterpreted the basis of your objection. At this point I'll need you to explain your objection in more detail if you wish me to find the appropriate literature addressing your concerns. Do you believe that the number of resamplings was too low in Blount's paper? That the analysis should have been performed with a software package other than Statistics101? Some other procedural issue? Some issue of interpretation?

::::The statistical problem that Blount must address is straightforward: given a distribution of mutant cultures that ''appears'' to be skewed toward the higher generations, what is the probability that this same amount of skew (or a greater degree) could arise by chance, given the null hypothesis that every generation is equally likely to produce a mutant? Interestingly, in the case of the first replay experiment, the total number of ways to randomly select (equal probability, no replacement) four cultures from seventy-two is 72x71x70x69, or 24,690,960. This number is small enough that a program can brute-force-calculate the 'mean generation number' of ''all possible'' combinations of four cultures in a reasonable amount of time. An experimentally-derived 'mean generation number' can be checked against this exhaustive list, and the number of means equal to or larger than the experimental mean can be found exactly. Converting this number to a percentage of 24,690,960 provides an exact p-value for any given experimental 'mean generation number'. This exhaustive approach is different than the Monte Carlo technique, in that ''all possible'' outcomes are examined, rather than a ''random subset'' of all possible outcomes. For the first replay experiment, it provides a way to independently check Blount's Monte Carlo results. This approach is not possible for the second and third replay experiments, in which the total number of possible combinations becomes impractically large: 340!/335! = 4.41 x10^12 and 2800!/2792! = 3.74 x 10^27, respectively.

::::I asked a colleague to run just such a brute-force program for me on the first replay data. I also ran several Monte Carlo simulations ('''not''' using Statsistics101) with Blount's data, using twenty-five million, one hundred million, and 493,819,200 resamplings - note that this last is twenty times the number of all possible combinations of 4 samples drawn without replacement from 72. The p-values from the 25M, 100M, and 493M Monte Carlo resamplings (0.00844, 0.00846, and 0.00846, respectively) compare favorably with Blount's 1M value of 0.0085 and the non-Monte-Carlo brute-force exact calculation, which provides a p-value of 0.008457. Thus it appears that Blount's statistical results are confirmed by a ''non-Monte Carlo'' technique, at least for the first replay experiment.

:::::What do you get for the experiment two p-value using the method from the paper and at least ten million realizations? [[User:SJohnson|SJohnson]] 08:48, 25 March 2009 (EDT)

::::::For the second replay experiment, Blount reports that one million resamplings gives a p-value of 0.0007. When I run the Monte Carlo simulations, ten million resamplings give a p-value of 0.00060; one hundred million resamplings give a p-value of 0.00062, and one ''billion'' resamplings give a p of 0.00061.

::::::As to the brute-force method for the second replay: the 4.41x10^12 combinations of five cultures picked from 340 actually represents 'only' 36.8 billion unique combinations, since for the purposes of calculating a mean generation value, the ordering of the cultures does not matter: 0, 0, 0, 0, 10 gives the same mean as 10, 0, 0, 0, 0 and 0, 10, 0, 0, 0. With brute force, it turns out that out of the 36,760,655,568 unique combinations possible in the second replay, 22,536,306 have means that are greater than or equal to 32,100.

::::::22,536,306 / 36,760,655,568 = 0.000613 = the ''exact'' p-value derived from exhaustive evaluation rather than Monte Carlo.

::::::The third replay has 9.27 x 10^22 unique combinations; at a billion comparisons a minute it would take over 170,000,000 years to check them all.--[[User:ElyM|ElyM]] 12:36, 25 March 2009 (EDT)

::::::----------
::::::Here are pointers to the Statistics 101 package <ref>http://www.statistics101.net/statistics101web_000003.htm</ref> and the actual programs run through the package by Blount ''et al.'' <ref>http://myxo.css.msu.edu/ecoli/citrate2008/MCprograms.html</ref>. The stats package is written in Java and should run under most major operating systems. A 10 million trial run of the second experiment took a bit of time and yielded a p-value of 0.00061. Ten separate, one-million trial runs produced an average p-value of 0.00061 (std.dev=0.00002, n=10). The numbers averaged about 0.0006 (std.dev=0.0004 n=10) 5K size trials.--[[User:Argon|Argon]] 20:52, 25 March 2009 (EDT)

::::My intention is not to get caught up in a digression about Monte Carlo, though - I'd rather keep the focus on the fact that the main article should acknowledge that SJohnson is using chi-square in a way that violates accepted guidelines; this remains true whether Blount's analysis is valid or not.--[[User:ElyM|ElyM]] 12:13, 23 March 2009 (EDT)

== References ==
{{reflist}}

Talk:Significance of E. Coli Evolution Experiments

2009-03-26T00:52:06Z

Argon: /* Unreferenced Claims */

SJohnson, your assessment, while good in the utilization of the chi-squared test is unfortunately incorrect. The Monte Carlo resampling gives a more accurate p-value than the chi-squared. You may research the literature (i.e. publications in statistical mathematics, many pubs actualy compare Monte Carlo vs Chi Squared) to discover that this method is commonly used in advance statistical work and how it is more accurate than the chi-squared test.--[[User:Able806|Able806]] 17:00, 4 March 2009 (EST)

:It doesn’t make sense to compare the chi-square test, which is a specific statistical hypothesis test, to Monte Carlo methods, which can be used for anything from fluid motion modeling to p-value computations. You can use Monte Carlo methods to compute the p-values of the chi-square test!

:Monte Carlo methods involve the generation of random realizations. Your broad claim the Monte Carlo methods are “more accurate” than the chi-square test is obviously incorrect because the accuracy of Monte Carlo methods always depends on the number of random realizations generated. When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous.

:Which publications compare Monte Carlo to chi-square and show that the former is more accurate? Could you provide specific examples? Thanks. [[User:SJohnson|SJohnson]] 18:50, 4 March 2009 (EST)

:In furtherance of SJohnson's remarks with respect to rarely occurring events, the use of the basic Monte Carlo method is plainly incorrect for modeling a rarely occurring event, as the Lenski paper did. This has long been pointed out in [[Flaws in Richard Lenski Study]]. I know [[evolutionists]] will never admit a flaw in anything promoting their pet theory, but this (and other) flaws in that paper is undeniable.

:Watch how evolutionists defended obvious errors in the Lenski paper, and then realize why the [[Piltdown Man]] fraud was taught for 40 years without evolutionists admitting it was a hoax.--[[User:Aschlafly|Andy Schlafly]] 09:55, 5 March 2009 (EST)

::Andy, how exactly is the Monte Carlo method incorrect to use in this case? I have seen it used in publications with much smaller datasets.--[[User:Able806|Able806]] 10:29, 5 March 2009 (EST)

:::Able806, I'm interested in looking at the publications you mentioned that use Monte Carlo methods to analyze small data sets. Could you provide some examples? Thanks. [[User:SJohnson|SJohnson]] 16:41, 5 March 2009 (EST)

::::SJohnson, here are two papers, [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WH8-45RFJ1J-19&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=1ad95954654bb97b17e474ce6b469f6e 1] and [http://cat.inist.fr/?aModele=afficheN&cpsidt=787963 2]. Most are in chemistry and genetics where you find the observed to be much smaller and have to use the MCM. You can search on the subject as well and find that how Lenski performed the test is the standard for microbiological genetic analysis.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

:::::Those papers have nothing to do with hypothesis testing. One is an archeology paper. To be blunt, it seems like you’re just doing internet searches on “Monte Carlo” to find these links. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

::::::SJohnson, actually they do, did you read the papers? If so you would see how they used the MCM for their data analysis of small data sets, which indeed was hypothesis testing and answers you inquiry about publications that use MCM for small data set analysis. If you wish I can try to track down some actual mathematical publications, however, I am not as familiar with mathematical journals as I am with science/medical journals (not knowing which mathematical journals are acceptable). I am assuming that you have a background in math and possibly access to mathematical journals, therefore if you know the reputable ones I can do the leg work.
::::::I believe the thing that needs to be looked at is there truly a problem with the choice of test and if so what is an alternative. Bayesian might be an option but seems to be difficult to employ for this situation.--[[User:Able806|Able806]] 12:36, 12 March 2009 (EDT)

:::Able806, you still seem to miss the point about how inappropriate the Monte Carlo method (as used in the Lenski paper) is for evaluating rarely occurring events. You need to open your mind to be productive. If you simply cling to a view that Lenski (who I don't think has any meaningful education in statistics) must somehow be right, then you're not going to make any progress in understanding the flaws.--[[User:Aschlafly|Andy Schlafly]] 17:07, 5 March 2009 (EST)

::::Andy, you still have not answered what you find inappropriate about his use of the Monte Carlo method? I am a reasonable person and with evidence I do have an open mind. I provided examples last week, with a working model, showing that Monte Carlo is better than the chi-square in this case. I have also shown where the Chi-Square was inappropriate due to the occurrence size as well. So if you have any evidence that Monte Carlo should not be used in the way that Lenski used please let it be shown.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

Sjohnson, I believe you just proved my point. In the literature of mean and covariance structure analysis, non-central chi-square distribution is commonly used to describe the behavior of the likelihood ratio statistic under alternative hypothesis; it is widely believed that the non-central chi-square distribution is justified by statistical theory. Actually, when the null hypothesis is not trivially violated, the non-central chi-square distribution cannot describe the LR statistic well even when data are normally distributed and the sample size is large. Monte Carlo results compare the strength of the normal distribution against that of the non-central chi-square distribution. In an association analysis comparing cases and controls with respect to allele frequencies at a highly polymorphic locus, a potential problem is that the conventional chi-squared test may not be valid for a large, sparse contingency table. Reliance on statistics with known asymptotic distribution is unnecessary, as Monte Carlo simulations can be performed to estimate the significance level of the test statistic.

Here is a [http://faculty.vassar.edu/lowry/chi_beta.html link] to a great page the provides an interactive example as to why the Chi Squared test would provide poor results compared to the Monte Carlo in relation to the Lenski data workup.

Something you may have overlooked was that the data set is actually too small to use the chi square method correctly. It is often accepted that is any of the analyzed data falls under 10 for a particular cell of the data set then the Yates correction needs to be applied; unfortunately the Yates correction can over correct thus skewing the p-value. Lenksi seemed to understand this by supporting his Monte Carlo p-value results with the Fisher z-transformation p-value.

I hope this helps.--[[User:Able806|Able806]] 10:27, 5 March 2009 (EST)

:I’m still waiting to hear which literature says that “Monte Carlo resampling” is “more accurate than the chi-squared test”. The page mentioned above [http://faculty.vassar.edu/lowry/chi_beta.html] is a discussion of why statisticians “fail to reject the null” rather than “accepting the null” when the p-value is above 0.05 or so. The page says nothing about superiority of Monte Carlo methods. Why were alternate hypothesis distributions mentioned? Only the null hypothesis distribution is used to calculate a p-value. Yates’s correction is for 2x2 contingency tables [http://en.wikipedia.org/wiki/Yates%27_correction_for_continuity]. It doesn’t apply in this case. Finally, what the heck do “covariance structure analysis” and “allele frequencies at a highly polymorphic locus” have to do with this problem? [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::SJohnson, I am looking for this paper for you, I cited it for one of my past publications dealing with allele frequencies (I believe it came from the Duke Biostatistics group). To answer your question about allele frequencies, that is the issue at hand, more about the genetics than the math, but it is the item being studied. So you stated that Yates can not be used and statistics says the number of occurrences is too small to evaluate using the Chi-Squared test so what would you recommend instead of the Monte-Carlo Method?

:Regarding the "Fisher z-transformation p-value" from the paper, garbage in garbage out. If the p-values were bad to begin with, then why would a combination of them be meaningful? [[User:SJohnson|SJohnson]] 10:49, 9 March 2009 (EDT)

::You are assuming that p-values are wrong based on a test that is inappropriate in this case due to data limitations. Did you perform a z-transformation on the chi-squared for the three data groups?--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

::You asked about the “Fisher z-transformation p-value”. The z-transformation test and Fisher’s method are actually two different things (see Whitlock's 2005 paper - Ref. 49 in Blount et al.). But no, I haven’t tried either. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::There's a large literature on various kinds of Monte Carlo test, a very short summary of which is that they're inevitably more accurate than parametric tests (e.g. F, t, chi-squared, etc) because they don't make assumptions about the distribution of the data under the null hypothesis. See for example ''Introduction to the Bootstrap'' by B. Efron and R. Tibshirani and ''The Jack-knife, the Bootstrap and Other Resampling Plans'', also by Efron. They're certainly applicable to small datasets and their accuracy is really only limited by the number of samples you care to take. E.g. 1000 M-C samples would give you a pretty accurate idea about significance at the alpha<1% level (That book should answer SJohnson's questions of 18:50 on 4/3/09 and 16:38 on 5/3/09 about accuracy and Aschalfly's comment of 17:07 on 5/3/09 about appropriateness of Monte Carlo tests.) [[User:FredFerguson|FredFerguson]] 16:53, 11 March 2009 (EDT)

::::Your claim that Monte Carlo methods are “inevitably more accurate” than other tests is obviously wrong because the accuracy of MC methods always depends on the number of realizations used. You should have written <math>\alpha=1\%</math>, not <math>\alpha<1\%</math>. If 1,000 random realizations are generated, the number of realizations above the true <math>\alpha=1\%</math> level is binomial with mean 10 and variance about 10. Thus, the standard deviation of the MC estimate is >0.003. In this example, a Monte Carlo p-value could be off by 30% and still be within a standard deviation. Is that really “pretty accurate”?

::::Using one million MC realizations (as done in the paper) at the <math>\alpha=0.001</math> level means the standard deviation is about 3%. The paper reported a p-value of less than 0.001 (experiment two). It wouldn’t surprise me to find out that the experiment two p-value for the flawed test is off because only one million realizations were used. My original statement, “When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous” is correct. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::::You're talking about miniscule differences in the accuracy of a test. 0.013 isn't very different from 0.007. In either case, it's very unlikely the experimenter would have obtained that result if the null hypothesis were true. If you're bothered about differences in P-values to the third decimals (which would make you unusual!), just run more MC realisations, that's all. Not really a problem. [[User:FredFerguson|FredFerguson]] 11:53, 12 March 2009 (EDT)

There’s still confusion about the difference between test statistics and Monte Carlo methods. Before you find a Monte Carlo estimate of a p-value, you need to select a test statistic to reduce the data set to a scalar. I am interested in hearing which test statistic you believe should be used in place of the chi-square test and why. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

----

Quick question for SJohnson: How many degrees of freedom did you choose when calculating the p-value? I'd like to know upon what condition you base that number. Thanks.--[[User:Argon|Argon]] 11:05, 5 March 2009 (EST)

:The degree of freedom for a contingency table is rows minus one times columns minus one. That is, <math> (r-1)(c-1) </math>. Here’s a pretty good tutorial I came across: [http://faculty.uncfsu.edu/dwallace/lesson%2020.pdf]. For the experiments from [http://www.pnas.org/content/105/23/7899.full.pdf], the DOFs are 11, 11, and 13. For experiment one, the chi-square test statistic is
::<math>
X^2
=\sum\limits_i\sum\limits_j
\frac{\left(n_{i,j}-E\left[n_{i,j}\right]\right)^2}
{E\left[n_{i,j}\right]}
</math>
::<math>
=\frac{\left(0-1/3\right)^2}{1/3}
+\frac{\left(6-17/3\right)^2}{17/3}
+\frac{\left(0-1/3\right)^2}{1/3}
+\ldots+
</math>
::<math>
+\frac{\left(2-1/3\right)^2}{1/3}
+\frac{\left(4-17/3\right)^2}{17/3}
</math>
::<math>
\approx
14.82
</math>
:where <math>n_{i,j}</math> is the observed value and <math>E\left[n_{i,j}\right]</math> is the expected null hypothesis value. So if you have MS Excel, another way to arrive at the p-value of 0.19 is to type “=CHIDIST(14.82,11)” into a cell. Cheers! [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::OK, thanks for the info. From what I'd calculated and looked up in tables, the numbers seemed close to a df=11 for a chi-square of ~14. (Aside: With terms having 17/3 in the denominator in the figures above, were you using the test of independence? I was using Pearson's test for [http://en.wikipedia.org/wiki/Pearson%27s_chi-square_test#Test_for_fit_of_a_distribution fit of a distribution] which returns a chi-squared value of 14 and roughly matched the p-values you reported, assuming the df was 11).

::Also, the first sentence of the article reads: "Blount, Borland, and Lenski[1] claimed that a key evolutionary innovation was observed during a laboratory experiment. That claim is false." A small correction: There were several claims in the paper. The 'key evolutionary innovation' was acquiring the ability to utilize citrate as a food source. That claim was demonstrated multiple times. The claim, which pertains to this statistics discussion was that the Cit+ phenotype arose in a multi-step process, first requiring a rare, pre-adaptive mutation before additional mutation(s) lead to the subsequent development of citrate utilization.--[[User:Argon|Argon]] 20:46, 5 March 2009 (EST)

:::My biology-degreed wife assures me that mutation does not necessarily mean that evolution occurred. What the paper claimed is that evolution (a “key innovation”) occurred in the lab. The key innovation supposedly increased the mutation rate. In the experiments, the observed mutation rate increased after generation 31,000, but not enough to make a statistically significant claim that the rate is not constant. The analysis in the paper was similar to flipping a coin ten times, counting six heads and claiming that the coin must be biased against tails. In reality, there’s nothing surprising about a fair coin producing slightly more of one outcome than the other. Just like there's nothing surprising about there being slightly more mutations in later generations than early generations given the null hypothesis (constant mutation rate). [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

>>Inserting a later comment first<<
SJohnson, the paper's title is: "Historical contingency '''and the evolution of a key innovation''' in an experimental population of ''Escherichia coli''" As I mentioned earlier, the key innovation is the evolution of the Cit+ phenotype and not the timing or rate of its acquisition. And yes, it *is* evolution (call it microevolution, if you wish). Blount et al went on further to speculate how this evolutionary innovation arose and they proposed the historical contingency hypothesis in which 'pre-adaptive' mutations were required before the Cit+ phenotype developed. It is only this latter hypothesis that you are attempting to address with your chi-square analysis, not the fact that Cit+ mutants arose (which is the evolutionary innovation).--[[User:Argon|Argon]] 21:57, 18 March 2009 (EDT)

::::SJohnson, not to say anything about your wife, but has she had a 400 level molecular genetics course (most general biology degrees do not cover the detail unless they are specialized)? If so, she would have mentioned that if the mutation passes to the offspring and is selectively beneficial to the population then it is a step of evolution as along as the conditions continue through the sharing of the mutation with the population and the environment is such that reduces the growth rate of the non-transformed population. While not all mutations are signs that evolution occurred the mutations that pass to offspring and provide a benefit compared to other offspring are very strong indicators. In the case of this paper the population that evolved the cit+ was able to metabolize a chemical in their environment which allowed for an adaptation advantage compared to the non-transformed colonies.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

----

Let’s go back to the beginning. There appears to be confusion about the difference between test statistics and methods for computing p-values. As is noted at the beginning of the page [http://www.conservapedia.com/Significance_of_E._Coli_Evolution_Experiments], the fundamental problem with the paper is that it used a flawed test statistic, not that it used Monte Carlo methods to find the p-value for that flawed statistic.

Every hypothesis test uses a test statistic to reduce the data to a single number. The p-value for the test statistic can be calculated analytically (as I’ve done for the chi-square test statistic) or by Monte Carlo methods. In the paper, Monte Carlo methods were used to compute the p-value of the “mutation generation” test statistic. The key problem with the analysis from the paper is that it doesn’t work to use a weighted average to test for variations in mutation rate. This is like trying to use the sample variance to test for an increase in the mean in Gaussian-distributed data. A statistic should be selected based on the null and alternate hypothesis distributions of the data. The chi-square test (unlike the weighted average from the paper) is a reasonable choice for data that mutates at a constant rate under the null hypothesis, but mutates at varying rates under the alternate hypothesis.

Able806, you made a good point about the contingency table cell frequencies being relatively low, but were wrong when you said ”the data set is actually too small to use the chi square method correctly”. In the low cell frequency case the chi-square test is still effective, but the null hypothesis distribution of the chi-square statistic starts to look less like the chi-square distribution. Thus, p-values calculated using the chi-square distribution may be a bit off. However, Monte Carlo p-values are always imperfect as well because it's impossible to generate an infinite number of random realizations. There are imperfections in p-values generated by analytic and Monte Carlo methods. However, low cell frequencies does not explain the >20x and >2.5x differences between chi-square p-values and p-values from the paper for experiments one and three. The reason for those huge differences was the use of the flawed test statistic (“mutation generation”) in the paper. [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

:SJohnson, the chi-squared test is a valuable statistical tool, but the limitations of the test must be acknowledged. The chi-squared test can only produce valid results if the assumptions that underly the test are not violated. As an analogy, Newtonian models of motion fail to produce accurate results as velocities approach the speed of light; under those circumstances one must switch to a theory that accounts for relativistic effects.

:It seems that you have simply dismissed the [http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm widely-acknowledged] [http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm fact] that the [http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html chi-squared test] is [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236 inappropriate] for use in [http://www.graphpad.com/www/Book/Choose.htm situations] where n in any cell is [http://mysite.du.edu/~jcalvert/econ/chisquar.htm less] less than a [http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1 threshold] [http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html number]. Different authors set different thresholds, but all are well above the numbers seen in your chi-squared analysis - even the most liberal guidelines advise against the chi-squared test when any expected cell frequency is less than one or more than 20% of the table cells are less than 5; others require that expected values in all cells must be more than 5. With smaller amounts of data, the test is insensitive and errs on the side of rejecting the hypothesis. If you attempt your chi-squared statistical analysis with a program that is more sophisticated than MS Excel (as I did), you get an error message indicating that the results are invalid due to low expected cell counts.

:That issue aside, there are other reasons that the chi-squared test is inappropriate here. As the links above point out, the categories tested must be truly independent; one example is that you can't use the chi-squared test to compare age and ability to kick a field goal by testing the same experimental group twice, one year apart; you have to test one group of age A and a different group of age B. In the case of the Blount paper, the categories are not independent. Even if there were adequate numbers to address the low-expected-frequency problem, this would make the chi-squared an invalid test in this case.

:There are other significant problems with the use of the chi-squared test in this circumstance, but they can wait until you address these first major problems.--[[User:ElyM|ElyM]] 12:18, 11 March 2009 (EDT)

::Wackerly et al. says in general it’s assumed that the cell frequencies are above five so that the chi-square statistic (under the null) is approximately chi-square distributed (see p. 703). That book does not say chi-square test results are invalid if frequencies are five or less. Your example of a chi-square test warning message (it said "warning" not "error" as you stated) in Minitab [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236] said “approximation probably invalid” referring to the chi-square distribution approximation to the chi-square test statistic’s distribution. Your example did not say “chi-square test invalid”. I agree that when cell frequencies are low, the chi-square test statistic’s distribution starts to deviate from the chi-square distribution. I maintain that this deviation is not enough to explain the >2.5x and >20x differences in the chi-square test p-values and the p-values from the paper.

::As the numerous links in your post proved, the chi-square test is widely-used by statisticians. Can you give examples of statisticians using mean mutation generation as a test statistic? Also, did your software agree with the chi-square test p-values I presented? Thanks. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::Thank you for giving page references for Wackerly; however it seems we have different editions, since page 703 in my copy (5th ed, 1996) does not deal with chi-squared issues at all. My copy does state the following, on page 622: "Although the mathematical proof is beyond the scope of this text, it can be shown that, when n is large [chi-squared] will possess approximately a chi-square probability distribution in repeated sampling." Then, on page 624: "Experience has shown that cell counts [n sub i] should not be too small in order that the chi-square distribution provide an accurate approximation to the distribution of [chi squared]. As a rule of thumb we require that all expected cell counts equal or exceed 5, although Cochran (1952) has noted that this value can be as low as 1 for some situations." Wackerly then goes on, in the problems sections, to describe the use of the chi-squared test as a "violation of good statistical practice" when "some expected counts [are] <5."

:::It seems that you are already aware that the [chi-square] statistic under the null is no longer chi-square distributed for small n; this is precisely why the test should not be used under those conditions. I can claim to be able to accelerate a 1-kg mass to 10 times the speed of light by applying 1 N of force for 95 years by using F=ma and t= (vf-vi)/a. Plugging the numbers into those equations will produce the same result every time, but the answer is illegitimate because those equations are only valid under certain assumptions, which are violated as velocities approach the speed of light. Similarly, having a statistical program calculate a chi-squared value given the Blount data will produce a number result, but since the assumptions of the test are violated the result is not legitimate. Yes, if I put the Blount data in SAS 9.2, I get the same numerical answer as you do, but I also get the following message: "WARNING: >89% of the cells have expected counts less than 5. Chi-square may not be a valid test." You may argue that that's a warning, not an error; that's a semantic distinction. The reason that the program says that it MAY not be valid is that the chi-squared test skews in the direction of being too conservative at low n values; the test has an acceptable rate of false positives but an unacceptably high rate of false negatives. Comparing the results of the Monte Carlo and chi-squared results in this case is like comparing the results of Newtonian and relativistic equations of motion: they can produce very different results from the same input data.

::::For a finite amount of data, the chi-square statistic is never chi-square distributed under the null. The p-values are always approximate regardless of cell frequencies. The approximation becomes more accurate as the amount of data increases, but I don’t believe that this inaccuracy will change p-values that are about 0.2 (for experiments 1 and 3) into statistically significant p-values. How much do you expect the p-values to change if an exact computation is used in place of the chi-square distribution approximation? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::Your last paragraph has a major non sequitur in it: yes, many statisticians use the chi-square test. As long as the assumptions of the test are not violated, it is a valuable tool. That has nothing to do with the validity of using mean mutation generation as a test statistic. 'Mean number of werewolf attacks in Mumbai in the week centered on the new moon, by month, from 1654 to 1798' is a valid test statistic. I am quite sure that it has never been used in a peer-reviewed paper before. That does not mean that I can't perform valid statistical tests on that statistic. If, however, the incorrect test is applied, the results of the analysis will be flawed. Papers apply a (relatively small) standard repertoire of valid tests to a (potentially infinite) number of test statistics. The particular test statistic used in a paper may never have been used before and may never be used again; that does not address the validity of the analysis. In Blount's case, the test is the Monte Carlo analysis, which is also "widely-used by statisticians".

::::There are an infinite number of ways to reduce a data set to a single number. However, it’s foolish to think every method would be effective. I gave an example of a flawed test statistic in an earlier post [http://www.conservapedia.com/index.php?title=Talk%3ASignificance_of_E._Coli_Evolution_Experiments&diff=635070&oldid=634987]. Another example of a flawed test statistic is the one used in the paper because it does not always detect deviations from the null hypothesis (see: [[Significance of E. Coli Evolution Experiments#Test Statistics]]).

::::Test statistics are typically derived. The likelihood ratio test is a common method used to derive them. The chi-square test for independence is an approximation to the LRT. Where is the derivation saying that mean mutation generation is an appropriate test statistic for this problem? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::We still haven't touched on the issue of the categories not being independent, which by itself is sufficient to invalidate the chi-squared technique. I'm new to this site, so I'm unsure as to the etiquette of making changes to the articles of another person - but the article here should at the very least mention that the chi-square test is being used here in a manner that violates its underlying assumptions in at least two fundamental ways, and the results are therefore suspect.--[[User:ElyM|ElyM]] 17:34, 12 March 2009 (EDT)

:::::When generating random realizations of experiment outcomes, the authors assumed that the total number of mutants was fixed. Thus the paper assumed the numbers of mutants per generation are statistically dependent. Does this seem like a realistic model, or do you think that if the experiments were recreated that the total number of mutants could vary? For example, if experiment one were recreated, would the total number of mutants always be exactly four? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::: It looks to me as though SJohnson has misinterpreted the application of the chi-squared test in quite a fundamental way. His/her analysis of Blount's data are therefore close to meaningless, regardless of whether the test used by Blount is appropriate or not. In my opinion, the entire page should therefore be deleted. [[User:FredFerguson|FredFerguson]] 08:18, 13 March 2009 (EDT)

::::: "Fred", perhaps you mistakenly think this is Wikipedia, where [[censorship]] and deletion of pages for ideological reasons are common. Not here.--[[User:Aschlafly|Andy Schlafly]] 10:23, 14 March 2009 (EDT)

:::::: Umm... I'm suggesting deletion for mathematical reasons, not ideological reasons. Using an argument filled with mathematical errors to try to support your case only detracts from your credibility. [[User:FredFerguson|FredFerguson]] 10:38, 14 March 2009 (EDT)

:::::: Actually, I think correction is better than deletion. So that's what I've done. [[User:FredFerguson|FredFerguson]] 11:01, 14 March 2009 (EDT)

::::::: I find no credibility in your denial of having ideological reasons.--[[User:Aschlafly|Andy Schlafly]] 11:04, 14 March 2009 (EDT)

== Misinterpretation of test ==

SJohnson, Your analysis misinterprets the test. You say the null hypothesis is that this mutation cannot happen. They saw a mutation (4 mutations, in fact, in the data set you show) so the null hypothesis (as you state is) is disproved. That's perfectly straightforward.

I don't know what the "mean mutation generation" test is but you're doing when you apply a chi-squared test to this dataset is to test if the mutations are evenly distributed throughout the generations. Your test says they are, so there's no strong evidence to suppose that mutations are likely to occur in one generation rather than another in the series of tests. Blount's test says thay aren't, so it's more likely that the mutation will occur later in the series of tests. I can't tell which test is right without knowing more about the test that Blount used.

But that point (the foregoing paragraph) has no bearing at all on the null hypothesis, as you describe it. The mutation appeared, so that means the hypothesis that the mutation can't happen is disproved. Very simple. [[User:FredFerguson|FredFerguson]] 21:10, 8 March 2009 (EDT)

:I never said that “the null hypothesis is that this mutation cannot happen”. The chi-square test statistic I'm using wouldn’t be defined if the null hypothesis mutation rate was zero because the <math>E\left[n_{i,j}\right]</math> term in the denominator of the statistic (see above equation) would be zero.

:The test statistic from the paper is the average of the generation numbers of observed mutations. For experiment one this number is
::<math>
\frac{1}{4}\left(30500+31500+2\times32500\right)= 31750.
</math>
:The same number is shown in Table 2 of the paper. [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

:: SJohnson, the way you're calculating the chi-squared statistic implies that you're testing the null hypothesis of a constant mutation rate over time against an alternative hypothesis of a mutation rate which varies over time. [[User:FredFerguson|FredFerguson]] 11:02, 9 March 2009 (EDT)

As it currently stands, the article makes the following statement: "The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown." This misstates the null hypothesis of the paper, which is elaborated in the Introduction section of the paper, and repeated in the section '''Statistical Analysis of the Replay Experiments''':
<blockquote>
For each experiment, we compared the observed mean generation of those clones that yielded Cit+ variants to the mean expected under the null hypothesis that clones from all generations have equal likelihood. The null thus corresponds to the rare-mutation hypothesis laid out in the Introduction."
Block quote
</blockquote><ref>www.pnas.org/cgi/reprint/105/23/7899.pdf</ref>

The article also continues to describe 'mean mutation generation' as a ''test'' rather than a ''statistic'' to which the ''Monte Carlo test'' was applied.--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

I'm a bit confused about why the Chi-squared test, which we're told compares the results to a null hypothesis of a constant mutation rate, seems insensitive to which generations the Cit+ mutations are found. Instead, the chi-square test seems only to be evaluating whether the frequencies of Cit+ mutations in any particular generation are 'expected'. Thus the test is asking whether finding a distribution (e.g. in the first experiment) across nine periods that have no mutations, two periods that have one mutation and one period with two mutations is a statistically significant deviation from what you'd expect of the mutations were randomly distributed. The number returned from the function is the same regardless of the order of Cit+ results. The number of mutations per bin is not the only question being asked. Instead it's the '''order and temporal distribution''' of Cit+ mutants that the analyses probably need to confront. It's not whether one can get nine no-mutants, two single mutants and one double-mutant result, it's a matter of '''when''' they occur and whether that distribution affects the significance of the results. Blount's hypothesis is that mutations should appear later in the experiment. When formulating a suitable null hypothesis, wouldn't one want to take the timing of Cit+ mutants into consideration too?--[[User:Argon|Argon]] 22:25, 18 March 2009 (EDT)

==Unreferenced Claims==

I deleted the claim that mean mutation generation is an appropriate test statistic because no reference was produced that back that claim. No reference was provided to back the claim that the chi-square test p-values are always conservative, either. [[User:SJohnson|SJohnson]] 12:57, 14 March 2009 (EDT)

: The reference is Everitt. I'll check I put it in the right place. [[User:FredFerguson|FredFerguson]] 13:30, 14 March 2009 (EDT)

There was a typo in my edit summaries on the talk page and the main page. I meant to say "Removed unsupported claims" rather than "Removed supported claims". [[User:SJohnson|SJohnson]] 13:13, 14 March 2009 (EDT)

I do not believe that anyone has claimed that 'chi-square test p-values are ''always'' conservative'. The claim that has been made is that ''under certain circumstances'', namely low n and low individual cell values, the chi-square test is an invalid test; that under those circumstances the power of the test is low and it becomes impossible to reject the null hypothesis even when it is false. You may have missed the pertinent sections in my links above, so I will directly quote the relevant sections. All the quoted sections refer to chi-square testing in particular. Any bolding below is mine.

:This edit claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639379]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

<blockquote>
"Assumptions: 

Even though a nonparametric statistic does not require a normally distributed population, there still are some restrictions regarding its use. 

1. Representative sample (Random) 
2. The data must be in frequency form (nominal data) or greater. 
3. The individual observations must be independent of each other. 
4. '''Sample size must be adequate. In a 2 x 2 table, Chi Square should not be used if n is less than 20. In a larger table, no expected value should be less than 1, and not more than 20% of the variables can have expected values of less than 5'''. 
5. Distribution basis must be decided on before the data is collected. 
6. The sum of the observed frequencies must equal the sum of the expected frequencies."
</blockquote>
<ref>http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm
</ref>

<blockquote>
"Assumptions:
* Random sample data are assumed. As with all significance tests, if you have population data, then any table differences are real and therefore significant. If you have non-random sample data, significance cannot be established, though significance tests are nonetheless sometimes utilized as crude "rules of thumb" anyway.
* A sufficiently large sample size is assumed, as in all significance tests. '''Applying chi-square to small samples exposes the researcher to an unacceptable rate of Type II errors. There is no accepted cutoff. Some set the minimum sample size at 50, while others would allow as few as 20'''. Note chi-square must be calculated on actual count data, not substituting percentages, which would have the effect of pretending the sample size is 100.
* '''Adequate cell sizes are also assumed. Some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count'''. When this assumption is not met, Yates' correction is applied.
* Independence. Observations must be independent. The same observation can only appear in one cell. '''This means chi-square cannot be used to test correlated data (ex., before-after, matched pairs, panel data)'''.
* Similar distribution. Observations must have the same underlying distribution.
* Known distribution. The hypothesized distribution is specified in advance, so that the number of observations that are expected to appear each cell in the table can be calculated without reference to the observed values. Normally this expected value is the crossproduct of the row and column marginals divided by the sample size.
* Non-directional hypotheses are assumed. Chi-square tests the hypothesis that two variables are related only by chance. If a significant relationship is found, this is not equivalent to establishing the researcher's hypothesis that A causes B, or that B causes A.
* Finite values. Observations must be grouped in categories.
* Normal distribution of deviations (observed minus expected values) is assumed. Note chi-square is a nonparametric test in the sense that is does not assume the parameter of normal distribution for the data -- only for the deviations.
* Data level. No assumption is made about level of data. Nominal, ordinal, or interval data may be used with chi-square tests."
</blockquote>
<ref>http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm</ref>

<blockquote>
"Assumptions: 
-None of the expected values may be less than 1 
-No more than 20% of the expected values may be less than 5"</blockquote> <ref>http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html</ref>

<blockquote>
"When performing a chi-square test, your data must satisfy important assumptions. Although these assumptions may be stated differently in different textbooks, they generally assert that: 
1)The sample must be randomly drawn from the population 
'''2)The sample size, n, must be large enough so that the expected cell count in each cell is greater than or equal to 5.''' 
Both assumptions must be met in the process of collecting your data, and violations of the second assumption will appear in the Minitab output when you run the analysis. 
... 
'''You may wonder why the second assumption is necessary for performing the chi-square test. The second assumption arises because the distribution of counts under the null hypothesis is multinomial, and the normal distribution can be used to approximate the multinomial distribution if the sample size is sufficiently large and the probability parameters aren't too small. It can be shown via the Central Limit Theorem that the multinomial distribution converges to the normal distribution as the sample size approaches infinity; however, there is no easy way to show mathematically how and when the convergence fails.'''"
</blockquote>
<ref>http://www.minitab.com/support/docs/Answers/Chi-Square%20Test%20Assumptions.pdf</ref>

<blockquote>
"'''The chi-square test is simpler to calculate but yields only an approximate P value. ... You should definitely avoid the chi-square test when the numbers in the contingency table are very small (any number less than about six)'''."
</blockquote> <ref>http://www.graphpad.com/www/Book/Choose.htm</ref>

<blockquote>
"The most important things to remember to get a valid χ2 test are that the expected values are not too small in any bin (certainly 5 or more), and that the degrees of freedom are properly evaluated. '''Unless you have a very large amount of data, the test is not very sensitive and errs on the side of safety. If you get a significant result, however, it is not likely to be wrong.'''"
</blockquote> <ref>http://mysite.du.edu/~jcalvert/econ/chisquar.htm</ref>

<blockquote>
"The critical assumptions of the chi-square test for k independent samples are similar to those for the chi-square test for two independent samples. ... 
'''4. No more than 20% of the cells may have expected frequencies of less than 5, and no cell should have an expected frequency of less than 1. 
The rule given in Assumption 4 is particularly important for a contingency table that is larger than 2X2'''"
</blockquote> <ref>http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1
</ref>

<blockquote>
"Special problems with small expected cell frequencies for the chi-square test: 
The chi-square test involves using the chi-square distribution to approximate the underlying exact distribution. The approximation becomes better as the expected cell frequencies grow larger, and '''may be inappropriate for tables with very small expected cell frequencies.''' 
'''For tables with expected cell frequencies less than 5, the chi-square approximation may not be reliable. A standard (and conservative) rule of thumb (due to Cochran) is to avoid using the chi-square test for tables with expected cell frequencies less than 1, or when more than 20% of the table cells have expected cell frequencies less than 5.''' 
Another rule of thumb (due to Roscoe and Byars) is that the average expected cell frequency should be at least 1 when the expected cell frequencies are close to equal, and 2 when they are not. (If the chosen significance level is 0.01 instead of 0.05, then double these numbers.) 
Koehler and Larntz suggest that if the total number of observations is at least 10, the number categories is at least 3, and the square of the total number of observations is at least 10 times the number of categories, then the chi-square approximation should be reasonable. 
Care should be taken when cell categories are combined (collapsed together) to fix problems of small expected cell frequencies. Collapsing can destroy evidence of non-independence, so a failure to reject the null hypothesis for the collapsed table does not rule out the possibility of non-independence in the original table. 
'''As with most statistical tests, the power of the chi-square test increases with a larger number of observations. If there are too few observations, it may be impossible to reject the null hypothesis even if it is false.'''"
</blockquote> <ref>http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html</ref>--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

: Thanks for this really excellent contribution, ElyM. The only thing I'd like to add is in relation to your initial statement, "I do not believe that anyone has claimed that 'chi-square test p-values are always conservative'". The question of whether a test is conservative in a particular situation is probabilistic. One can determine whether a test is likely to generate a p-value which is too high in a particular situation (e.g. for a chi-squared test, when there are lots of small expected values) but one needs an exact test (such as an appropriate Monte Carlo randomisation test) to determine whether the p-value in any ''particular'' test is in fact excessively high.

::This edit also claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639373]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

: I hope careful reading of your very clear description will put SJohnson's mind at rest on this subject. [[User:FredFerguson|FredFerguson]] 18:11, 14 March 2009 (EDT)

ElyM, you've provided nothing to address the basic flaw that "The paper incorrectly applied a Monte Carlo resampling test to exclude the null hypothesis for rarely occurring events." See [[Flaws in Lenski Study]]. Also, do not impose your view on the content page until after SJohnson has had an opportunity to respond to your posting. As to "Fred", his put-downs are getting tiresome and I'm going to review his edit pattern now to see if he's been contributing anything of value to this site.--[[User:Aschlafly|Andy Schlafly]] 14:06, 15 March 2009 (EDT)

::Mr. Schlafly, per your request I have not added anything to the content page as SJohnson has not yet responded to my posts. Since all of my comments have been in regards to SJohnson's use of the chi-square test in this particular article, I'm not sure why you expect me to address Blout's use of Monte Carlo - that issue seems to be addressed on the [[Flaws in Lenski Study]] page. SJohnson has added a reformulation of the chi-square test for two possible outcomes, and stated that the chi-square test is at a minimum when all success probabilities are equal. He then extrapolates from this to claim that the chi-square test is an effective test for the data from Blount.

::The reformulation of the equations for two possible outcomes does not address the underlying problem that the chi-square test has universally accepted parameters outside of which it is considered an invalid test; I have provided references for these parameters and shown that the data from Blount lies outside them. None of the expected cells in SJohnson's analysis have values above one, and the total n is four. SJohnson's own reference states that the application of the chi-square test in this circumstance is a "violation of good statistical practice". Analogously, combining F=ma and t=(vf-vi)/a into t=(vf-vi)m/F and showing that t is a minimum when m approaches zero does not address the fact that those Newtonian equations do not apply as velocities approach the speed of light. The legitimacy of Blount's arguments cannot be determined by the application of illegitimate counterarguments. If SJohnson or others can point to references from the statistical literature that show that Blount has made methodological errors - as I have been able to do with SJohnson's chi-square analysis - I would welcome their input, and no doubt Conservapedia's other readers would as well, and this page would be greatly improved.

::I have not seen a rebuttal from SJohnson in the four days since my last post, although he has added new material to the content page since then. In light of this, I would appreciate some guidelines as to when it is appropriate for me to add my information and references to the content page. I can add citations from the primary mathematical literature if necessary, but in general I find that these are less helpful as they are not easily accessible by readers without access to academic libraries.--[[User:ElyM|ElyM]] 18:07, 18 March 2009 (EDT)

::: You say, "I'm not sure why you expect me to address Blout's use of Monte Carlo." The reason is obvious: the title of the content page is the "Significance of E. Coli Evolution Experiments." You haven't addressed the inappropriateness of using Monte Carlo simulations for assessing the significance rarely occurring events, which was central to Lenski's statistical claims. I suggest you address this flaw if you want to be taken seriously.--[[User:Aschlafly|Andy Schlafly]] 23:12, 18 March 2009 (EDT)

::::ASchlafly, again per your request, and based on your statement regarding the "inappropriateness of using Monte Carlo simulations for assessing the significance of rarely occurring events", I have spent the last several days reviewing the literature available to me on Monte Carlo and other resampling techniques, looking for ways in which Blount may have made a methodological error of the sort that SJohnson has made. I have been unable to find any examples of authors suggesting that Monte Carlo be avoided for low ''n'', or for events with low probability regardless of'' n'', much less providing specific cutoff numbers as are seen in the references that I provided for the chi-square test. Similarly, the technique that Blount used does not require/assume that categories are unrelated, as the chi-square test does. Of course, the absence of evidence is not evidence of absence, and I may have misinterpreted the basis of your objection. At this point I'll need you to explain your objection in more detail if you wish me to find the appropriate literature addressing your concerns. Do you believe that the number of resamplings was too low in Blount's paper? That the analysis should have been performed with a software package other than Statistics101? Some other procedural issue? Some issue of interpretation?

::::The statistical problem that Blount must address is straightforward: given a distribution of mutant cultures that ''appears'' to be skewed toward the higher generations, what is the probability that this same amount of skew (or a greater degree) could arise by chance, given the null hypothesis that every generation is equally likely to produce a mutant? Interestingly, in the case of the first replay experiment, the total number of ways to randomly select (equal probability, no replacement) four cultures from seventy-two is 72x71x70x69, or 24,690,960. This number is small enough that a program can brute-force-calculate the 'mean generation number' of ''all possible'' combinations of four cultures in a reasonable amount of time. An experimentally-derived 'mean generation number' can be checked against this exhaustive list, and the number of means equal to or larger than the experimental mean can be found exactly. Converting this number to a percentage of 24,690,960 provides an exact p-value for any given experimental 'mean generation number'. This exhaustive approach is different than the Monte Carlo technique, in that ''all possible'' outcomes are examined, rather than a ''random subset'' of all possible outcomes. For the first replay experiment, it provides a way to independently check Blount's Monte Carlo results. This approach is not possible for the second and third replay experiments, in which the total number of possible combinations becomes impractically large: 340!/335! = 4.41 x10^12 and 2800!/2792! = 3.74 x 10^27, respectively.

::::I asked a colleague to run just such a brute-force program for me on the first replay data. I also ran several Monte Carlo simulations ('''not''' using Statsistics101) with Blount's data, using twenty-five million, one hundred million, and 493,819,200 resamplings - note that this last is twenty times the number of all possible combinations of 4 samples drawn without replacement from 72. The p-values from the 25M, 100M, and 493M Monte Carlo resamplings (0.00844, 0.00846, and 0.00846, respectively) compare favorably with Blount's 1M value of 0.0085 and the non-Monte-Carlo brute-force exact calculation, which provides a p-value of 0.008457. Thus it appears that Blount's statistical results are confirmed by a ''non-Monte Carlo'' technique, at least for the first replay experiment.

:::::What do you get for the experiment two p-value using the method from the paper and at least ten million realizations? [[User:SJohnson|SJohnson]] 08:48, 25 March 2009 (EDT)

::::::For the second replay experiment, Blount reports that one million resamplings gives a p-value of 0.0007. When I run the Monte Carlo simulations, ten million resamplings give a p-value of 0.00060; one hundred million resamplings give a p-value of 0.00062, and one ''billion'' resamplings give a p of 0.00061.

::::::As to the brute-force method for the second replay: the 4.41x10^12 combinations of five cultures picked from 340 actually represents 'only' 36.8 billion unique combinations, since for the purposes of calculating a mean generation value, the ordering of the cultures does not matter: 0, 0, 0, 0, 10 gives the same mean as 10, 0, 0, 0, 0 and 0, 10, 0, 0, 0. With brute force, it turns out that out of the 36,760,655,568 unique combinations possible in the second replay, 22,536,306 have means that are greater than or equal to 32,100.

::::::22,536,306 / 36,760,655,568 = 0.000613 = the ''exact'' p-value derived from exhaustive evaluation rather than Monte Carlo.

::::::The third replay has 9.27 x 10^22 unique combinations; at a billion comparisons a minute it would take over 170,000,000 years to check them all.--[[User:ElyM|ElyM]] 12:36, 25 March 2009 (EDT)

::::::----------
::::::Here are pointers to the Statistics 101 package <ref>http://www.statistics101.net/statistics101web_000003.htm</ref> and the actual programs run through the package by Blount ''et al.'' <ref>https://myxo.css.msu.edu/ecoli/citrate2008/MCprograms.html</ref>. The stats package is written in Java and should run under most major operating systems. A 10 million trial run of the second experiment took a bit of time and yielded a p-value of 0.00061. Ten separate, one-million trial runs produced an average p-value of 0.00061 (std.dev=0.00002, n=10). The numbers averaged about 0.0006 (std.dev=0.0004 n=10) 5K size trials.--[[User:Argon|Argon]] 20:52, 25 March 2009 (EDT)

::::My intention is not to get caught up in a digression about Monte Carlo, though - I'd rather keep the focus on the fact that the main article should acknowledge that SJohnson is using chi-square in a way that violates accepted guidelines; this remains true whether Blount's analysis is valid or not.--[[User:ElyM|ElyM]] 12:13, 23 March 2009 (EDT)

== References ==
{{reflist}}

Talk:Significance of E. Coli Evolution Experiments

2009-03-19T02:25:06Z

Argon: /* Misinterpretation of test */

Talk:Significance of E. Coli Evolution Experiments

2009-03-19T01:58:23Z

Argon:

SJohnson, your assessment, while good in the utilization of the chi-squared test is unfortunately incorrect. The Monte Carlo resampling gives a more accurate p-value than the chi-squared. You may research the literature (i.e. publications in statistical mathematics, many pubs actualy compare Monte Carlo vs Chi Squared) to discover that this method is commonly used in advance statistical work and how it is more accurate than the chi-squared test.--[[User:Able806|Able806]] 17:00, 4 March 2009 (EST)

:It doesn’t make sense to compare the chi-square test, which is a specific statistical hypothesis test, to Monte Carlo methods, which can be used for anything from fluid motion modeling to p-value computations. You can use Monte Carlo methods to compute the p-values of the chi-square test!

:Monte Carlo methods involve the generation of random realizations. Your broad claim the Monte Carlo methods are “more accurate” than the chi-square test is obviously incorrect because the accuracy of Monte Carlo methods always depends on the number of random realizations generated. When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous.

:Which publications compare Monte Carlo to chi-square and show that the former is more accurate? Could you provide specific examples? Thanks. [[User:SJohnson|SJohnson]] 18:50, 4 March 2009 (EST)

:In furtherance of SJohnson's remarks with respect to rarely occurring events, the use of the basic Monte Carlo method is plainly incorrect for modeling a rarely occurring event, as the Lenski paper did. This has long been pointed out in [[Flaws in Richard Lenski Study]]. I know [[evolutionists]] will never admit a flaw in anything promoting their pet theory, but this (and other) flaws in that paper is undeniable.

:Watch how evolutionists defended obvious errors in the Lenski paper, and then realize why the [[Piltdown Man]] fraud was taught for 40 years without evolutionists admitting it was a hoax.--[[User:Aschlafly|Andy Schlafly]] 09:55, 5 March 2009 (EST)

::Andy, how exactly is the Monte Carlo method incorrect to use in this case? I have seen it used in publications with much smaller datasets.--[[User:Able806|Able806]] 10:29, 5 March 2009 (EST)

:::Able806, I'm interested in looking at the publications you mentioned that use Monte Carlo methods to analyze small data sets. Could you provide some examples? Thanks. [[User:SJohnson|SJohnson]] 16:41, 5 March 2009 (EST)

::::SJohnson, here are two papers, [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WH8-45RFJ1J-19&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=1ad95954654bb97b17e474ce6b469f6e 1] and [http://cat.inist.fr/?aModele=afficheN&cpsidt=787963 2]. Most are in chemistry and genetics where you find the observed to be much smaller and have to use the MCM. You can search on the subject as well and find that how Lenski performed the test is the standard for microbiological genetic analysis.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

:::::Those papers have nothing to do with hypothesis testing. One is an archeology paper. To be blunt, it seems like you’re just doing internet searches on “Monte Carlo” to find these links. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

::::::SJohnson, actually they do, did you read the papers? If so you would see how they used the MCM for their data analysis of small data sets, which indeed was hypothesis testing and answers you inquiry about publications that use MCM for small data set analysis. If you wish I can try to track down some actual mathematical publications, however, I am not as familiar with mathematical journals as I am with science/medical journals (not knowing which mathematical journals are acceptable). I am assuming that you have a background in math and possibly access to mathematical journals, therefore if you know the reputable ones I can do the leg work.
::::::I believe the thing that needs to be looked at is there truly a problem with the choice of test and if so what is an alternative. Bayesian might be an option but seems to be difficult to employ for this situation.--[[User:Able806|Able806]] 12:36, 12 March 2009 (EDT)

:::Able806, you still seem to miss the point about how inappropriate the Monte Carlo method (as used in the Lenski paper) is for evaluating rarely occurring events. You need to open your mind to be productive. If you simply cling to a view that Lenski (who I don't think has any meaningful education in statistics) must somehow be right, then you're not going to make any progress in understanding the flaws.--[[User:Aschlafly|Andy Schlafly]] 17:07, 5 March 2009 (EST)

::::Andy, you still have not answered what you find inappropriate about his use of the Monte Carlo method? I am a reasonable person and with evidence I do have an open mind. I provided examples last week, with a working model, showing that Monte Carlo is better than the chi-square in this case. I have also shown where the Chi-Square was inappropriate due to the occurrence size as well. So if you have any evidence that Monte Carlo should not be used in the way that Lenski used please let it be shown.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

Sjohnson, I believe you just proved my point. In the literature of mean and covariance structure analysis, non-central chi-square distribution is commonly used to describe the behavior of the likelihood ratio statistic under alternative hypothesis; it is widely believed that the non-central chi-square distribution is justified by statistical theory. Actually, when the null hypothesis is not trivially violated, the non-central chi-square distribution cannot describe the LR statistic well even when data are normally distributed and the sample size is large. Monte Carlo results compare the strength of the normal distribution against that of the non-central chi-square distribution. In an association analysis comparing cases and controls with respect to allele frequencies at a highly polymorphic locus, a potential problem is that the conventional chi-squared test may not be valid for a large, sparse contingency table. Reliance on statistics with known asymptotic distribution is unnecessary, as Monte Carlo simulations can be performed to estimate the significance level of the test statistic.

Here is a [http://faculty.vassar.edu/lowry/chi_beta.html link] to a great page the provides an interactive example as to why the Chi Squared test would provide poor results compared to the Monte Carlo in relation to the Lenski data workup.

Something you may have overlooked was that the data set is actually too small to use the chi square method correctly. It is often accepted that is any of the analyzed data falls under 10 for a particular cell of the data set then the Yates correction needs to be applied; unfortunately the Yates correction can over correct thus skewing the p-value. Lenksi seemed to understand this by supporting his Monte Carlo p-value results with the Fisher z-transformation p-value.

I hope this helps.--[[User:Able806|Able806]] 10:27, 5 March 2009 (EST)

:I’m still waiting to hear which literature says that “Monte Carlo resampling” is “more accurate than the chi-squared test”. The page mentioned above [http://faculty.vassar.edu/lowry/chi_beta.html] is a discussion of why statisticians “fail to reject the null” rather than “accepting the null” when the p-value is above 0.05 or so. The page says nothing about superiority of Monte Carlo methods. Why were alternate hypothesis distributions mentioned? Only the null hypothesis distribution is used to calculate a p-value. Yates’s correction is for 2x2 contingency tables [http://en.wikipedia.org/wiki/Yates%27_correction_for_continuity]. It doesn’t apply in this case. Finally, what the heck do “covariance structure analysis” and “allele frequencies at a highly polymorphic locus” have to do with this problem? [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::SJohnson, I am looking for this paper for you, I cited it for one of my past publications dealing with allele frequencies (I believe it came from the Duke Biostatistics group). To answer your question about allele frequencies, that is the issue at hand, more about the genetics than the math, but it is the item being studied. So you stated that Yates can not be used and statistics says the number of occurrences is too small to evaluate using the Chi-Squared test so what would you recommend instead of the Monte-Carlo Method?

:Regarding the "Fisher z-transformation p-value" from the paper, garbage in garbage out. If the p-values were bad to begin with, then why would a combination of them be meaningful? [[User:SJohnson|SJohnson]] 10:49, 9 March 2009 (EDT)

::You are assuming that p-values are wrong based on a test that is inappropriate in this case due to data limitations. Did you perform a z-transformation on the chi-squared for the three data groups?--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

::You asked about the “Fisher z-transformation p-value”. The z-transformation test and Fisher’s method are actually two different things (see Whitlock's 2005 paper - Ref. 49 in Blount et al.). But no, I haven’t tried either. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::There's a large literature on various kinds of Monte Carlo test, a very short summary of which is that they're inevitably more accurate than parametric tests (e.g. F, t, chi-squared, etc) because they don't make assumptions about the distribution of the data under the null hypothesis. See for example ''Introduction to the Bootstrap'' by B. Efron and R. Tibshirani and ''The Jack-knife, the Bootstrap and Other Resampling Plans'', also by Efron. They're certainly applicable to small datasets and their accuracy is really only limited by the number of samples you care to take. E.g. 1000 M-C samples would give you a pretty accurate idea about significance at the alpha<1% level (That book should answer SJohnson's questions of 18:50 on 4/3/09 and 16:38 on 5/3/09 about accuracy and Aschalfly's comment of 17:07 on 5/3/09 about appropriateness of Monte Carlo tests.) [[User:FredFerguson|FredFerguson]] 16:53, 11 March 2009 (EDT)

::::Your claim that Monte Carlo methods are “inevitably more accurate” than other tests is obviously wrong because the accuracy of MC methods always depends on the number of realizations used. You should have written <math>\alpha=1\%</math>, not <math>\alpha<1\%</math>. If 1,000 random realizations are generated, the number of realizations above the true <math>\alpha=1\%</math> level is binomial with mean 10 and variance about 10. Thus, the standard deviation of the MC estimate is >0.003. In this example, a Monte Carlo p-value could be off by 30% and still be within a standard deviation. Is that really “pretty accurate”?

::::Using one million MC realizations (as done in the paper) at the <math>\alpha=0.001</math> level means the standard deviation is about 3%. The paper reported a p-value of less than 0.001 (experiment two). It wouldn’t surprise me to find out that the experiment two p-value for the flawed test is off because only one million realizations were used. My original statement, “When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous” is correct. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::::You're talking about miniscule differences in the accuracy of a test. 0.013 isn't very different from 0.007. In either case, it's very unlikely the experimenter would have obtained that result if the null hypothesis were true. If you're bothered about differences in P-values to the third decimals (which would make you unusual!), just run more MC realisations, that's all. Not really a problem. [[User:FredFerguson|FredFerguson]] 11:53, 12 March 2009 (EDT)

There’s still confusion about the difference between test statistics and Monte Carlo methods. Before you find a Monte Carlo estimate of a p-value, you need to select a test statistic to reduce the data set to a scalar. I am interested in hearing which test statistic you believe should be used in place of the chi-square test and why. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

----

Quick question for SJohnson: How many degrees of freedom did you choose when calculating the p-value? I'd like to know upon what condition you base that number. Thanks.--[[User:Argon|Argon]] 11:05, 5 March 2009 (EST)

:The degree of freedom for a contingency table is rows minus one times columns minus one. That is, <math> (r-1)(c-1) </math>. Here’s a pretty good tutorial I came across: [http://faculty.uncfsu.edu/dwallace/lesson%2020.pdf]. For the experiments from [http://www.pnas.org/content/105/23/7899.full.pdf], the DOFs are 11, 11, and 13. For experiment one, the chi-square test statistic is
::<math>
X^2
=\sum\limits_i\sum\limits_j
\frac{\left(n_{i,j}-E\left[n_{i,j}\right]\right)^2}
{E\left[n_{i,j}\right]}
</math>
::<math>
=\frac{\left(0-1/3\right)^2}{1/3}
+\frac{\left(6-17/3\right)^2}{17/3}
+\frac{\left(0-1/3\right)^2}{1/3}
+\ldots+
</math>
::<math>
+\frac{\left(2-1/3\right)^2}{1/3}
+\frac{\left(4-17/3\right)^2}{17/3}
</math>
::<math>
\approx
14.82
</math>
:where <math>n_{i,j}</math> is the observed value and <math>E\left[n_{i,j}\right]</math> is the expected null hypothesis value. So if you have MS Excel, another way to arrive at the p-value of 0.19 is to type “=CHIDIST(14.82,11)” into a cell. Cheers! [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::OK, thanks for the info. From what I'd calculated and looked up in tables, the numbers seemed close to a df=11 for a chi-square of ~14. (Aside: With terms having 17/3 in the denominator in the figures above, were you using the test of independence? I was using Pearson's test for [http://en.wikipedia.org/wiki/Pearson%27s_chi-square_test#Test_for_fit_of_a_distribution fit of a distribution] which returns a chi-squared value of 14 and roughly matched the p-values you reported, assuming the df was 11).

::Also, the first sentence of the article reads: "Blount, Borland, and Lenski[1] claimed that a key evolutionary innovation was observed during a laboratory experiment. That claim is false." A small correction: There were several claims in the paper. The 'key evolutionary innovation' was acquiring the ability to utilize citrate as a food source. That claim was demonstrated multiple times. The claim, which pertains to this statistics discussion was that the Cit+ phenotype arose in a multi-step process, first requiring a rare, pre-adaptive mutation before additional mutation(s) lead to the subsequent development of citrate utilization.--[[User:Argon|Argon]] 20:46, 5 March 2009 (EST)

:::My biology-degreed wife assures me that mutation does not necessarily mean that evolution occurred. What the paper claimed is that evolution (a “key innovation”) occurred in the lab. The key innovation supposedly increased the mutation rate. In the experiments, the observed mutation rate increased after generation 31,000, but not enough to make a statistically significant claim that the rate is not constant. The analysis in the paper was similar to flipping a coin ten times, counting six heads and claiming that the coin must be biased against tails. In reality, there’s nothing surprising about a fair coin producing slightly more of one outcome than the other. Just like there's nothing surprising about there being slightly more mutations in later generations than early generations given the null hypothesis (constant mutation rate). [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

>>Inserting a later comment first<<
SJohnson, the paper's title is: "Historical contingency '''and the evolution of a key innovation''' in an experimental population of ''Escherichia coli''" As I mentioned earlier, the key innovation is the evolution of the Cit+ phenotype and not the timing or rate of its acquisition. And yes, it *is* evolution (call it microevolution, if you wish). Blount et al went on further to speculate how this evolutionary innovation arose and they proposed the historical contingency hypothesis in which 'pre-adaptive' mutations were required before the Cit+ phenotype developed. It is only this latter hypothesis that you are attempting to address with your chi-square analysis, not the fact that Cit+ mutants arose (which is the evolutionary innovation).--[[User:Argon|Argon]] 21:57, 18 March 2009 (EDT)

::::SJohnson, not to say anything about your wife, but has she had a 400 level molecular genetics course (most general biology degrees do not cover the detail unless they are specialized)? If so, she would have mentioned that if the mutation passes to the offspring and is selectively beneficial to the population then it is a step of evolution as along as the conditions continue through the sharing of the mutation with the population and the environment is such that reduces the growth rate of the non-transformed population. While not all mutations are signs that evolution occurred the mutations that pass to offspring and provide a benefit compared to other offspring are very strong indicators. In the case of this paper the population that evolved the cit+ was able to metabolize a chemical in their environment which allowed for an adaptation advantage compared to the non-transformed colonies.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

----

Let’s go back to the beginning. There appears to be confusion about the difference between test statistics and methods for computing p-values. As is noted at the beginning of the page [http://www.conservapedia.com/Significance_of_E._Coli_Evolution_Experiments], the fundamental problem with the paper is that it used a flawed test statistic, not that it used Monte Carlo methods to find the p-value for that flawed statistic.

Every hypothesis test uses a test statistic to reduce the data to a single number. The p-value for the test statistic can be calculated analytically (as I’ve done for the chi-square test statistic) or by Monte Carlo methods. In the paper, Monte Carlo methods were used to compute the p-value of the “mutation generation” test statistic. The key problem with the analysis from the paper is that it doesn’t work to use a weighted average to test for variations in mutation rate. This is like trying to use the sample variance to test for an increase in the mean in Gaussian-distributed data. A statistic should be selected based on the null and alternate hypothesis distributions of the data. The chi-square test (unlike the weighted average from the paper) is a reasonable choice for data that mutates at a constant rate under the null hypothesis, but mutates at varying rates under the alternate hypothesis.

Able806, you made a good point about the contingency table cell frequencies being relatively low, but were wrong when you said ”the data set is actually too small to use the chi square method correctly”. In the low cell frequency case the chi-square test is still effective, but the null hypothesis distribution of the chi-square statistic starts to look less like the chi-square distribution. Thus, p-values calculated using the chi-square distribution may be a bit off. However, Monte Carlo p-values are always imperfect as well because it's impossible to generate an infinite number of random realizations. There are imperfections in p-values generated by analytic and Monte Carlo methods. However, low cell frequencies does not explain the >20x and >2.5x differences between chi-square p-values and p-values from the paper for experiments one and three. The reason for those huge differences was the use of the flawed test statistic (“mutation generation”) in the paper. [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

:SJohnson, the chi-squared test is a valuable statistical tool, but the limitations of the test must be acknowledged. The chi-squared test can only produce valid results if the assumptions that underly the test are not violated. As an analogy, Newtonian models of motion fail to produce accurate results as velocities approach the speed of light; under those circumstances one must switch to a theory that accounts for relativistic effects.

:It seems that you have simply dismissed the [http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm widely-acknowledged] [http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm fact] that the [http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html chi-squared test] is [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236 inappropriate] for use in [http://www.graphpad.com/www/Book/Choose.htm situations] where n in any cell is [http://mysite.du.edu/~jcalvert/econ/chisquar.htm less] less than a [http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1 threshold] [http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html number]. Different authors set different thresholds, but all are well above the numbers seen in your chi-squared analysis - even the most liberal guidelines advise against the chi-squared test when any expected cell frequency is less than one or more than 20% of the table cells are less than 5; others require that expected values in all cells must be more than 5. With smaller amounts of data, the test is insensitive and errs on the side of rejecting the hypothesis. If you attempt your chi-squared statistical analysis with a program that is more sophisticated than MS Excel (as I did), you get an error message indicating that the results are invalid due to low expected cell counts.

:That issue aside, there are other reasons that the chi-squared test is inappropriate here. As the links above point out, the categories tested must be truly independent; one example is that you can't use the chi-squared test to compare age and ability to kick a field goal by testing the same experimental group twice, one year apart; you have to test one group of age A and a different group of age B. In the case of the Blount paper, the categories are not independent. Even if there were adequate numbers to address the low-expected-frequency problem, this would make the chi-squared an invalid test in this case.

:There are other significant problems with the use of the chi-squared test in this circumstance, but they can wait until you address these first major problems.--[[User:ElyM|ElyM]] 12:18, 11 March 2009 (EDT)

::Wackerly et al. says in general it’s assumed that the cell frequencies are above five so that the chi-square statistic (under the null) is approximately chi-square distributed (see p. 703). That book does not say chi-square test results are invalid if frequencies are five or less. Your example of a chi-square test warning message (it said "warning" not "error" as you stated) in Minitab [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236] said “approximation probably invalid” referring to the chi-square distribution approximation to the chi-square test statistic’s distribution. Your example did not say “chi-square test invalid”. I agree that when cell frequencies are low, the chi-square test statistic’s distribution starts to deviate from the chi-square distribution. I maintain that this deviation is not enough to explain the >2.5x and >20x differences in the chi-square test p-values and the p-values from the paper.

::As the numerous links in your post proved, the chi-square test is widely-used by statisticians. Can you give examples of statisticians using mean mutation generation as a test statistic? Also, did your software agree with the chi-square test p-values I presented? Thanks. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::Thank you for giving page references for Wackerly; however it seems we have different editions, since page 703 in my copy (5th ed, 1996) does not deal with chi-squared issues at all. My copy does state the following, on page 622: "Although the mathematical proof is beyond the scope of this text, it can be shown that, when n is large [chi-squared] will possess approximately a chi-square probability distribution in repeated sampling." Then, on page 624: "Experience has shown that cell counts [n sub i] should not be too small in order that the chi-square distribution provide an accurate approximation to the distribution of [chi squared]. As a rule of thumb we require that all expected cell counts equal or exceed 5, although Cochran (1952) has noted that this value can be as low as 1 for some situations." Wackerly then goes on, in the problems sections, to describe the use of the chi-squared test as a "violation of good statistical practice" when "some expected counts [are] <5."

:::It seems that you are already aware that the [chi-square] statistic under the null is no longer chi-square distributed for small n; this is precisely why the test should not be used under those conditions. I can claim to be able to accelerate a 1-kg mass to 10 times the speed of light by applying 1 N of force for 95 years by using F=ma and t= (vf-vi)/a. Plugging the numbers into those equations will produce the same result every time, but the answer is illegitimate because those equations are only valid under certain assumptions, which are violated as velocities approach the speed of light. Similarly, having a statistical program calculate a chi-squared value given the Blount data will produce a number result, but since the assumptions of the test are violated the result is not legitimate. Yes, if I put the Blount data in SAS 9.2, I get the same numerical answer as you do, but I also get the following message: "WARNING: >89% of the cells have expected counts less than 5. Chi-square may not be a valid test." You may argue that that's a warning, not an error; that's a semantic distinction. The reason that the program says that it MAY not be valid is that the chi-squared test skews in the direction of being too conservative at low n values; the test has an acceptable rate of false positives but an unacceptably high rate of false negatives. Comparing the results of the Monte Carlo and chi-squared results in this case is like comparing the results of Newtonian and relativistic equations of motion: they can produce very different results from the same input data.

::::For a finite amount of data, the chi-square statistic is never chi-square distributed under the null. The p-values are always approximate regardless of cell frequencies. The approximation becomes more accurate as the amount of data increases, but I don’t believe that this inaccuracy will change p-values that are about 0.2 (for experiments 1 and 3) into statistically significant p-values. How much do you expect the p-values to change if an exact computation is used in place of the chi-square distribution approximation? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::Your last paragraph has a major non sequitur in it: yes, many statisticians use the chi-square test. As long as the assumptions of the test are not violated, it is a valuable tool. That has nothing to do with the validity of using mean mutation generation as a test statistic. 'Mean number of werewolf attacks in Mumbai in the week centered on the new moon, by month, from 1654 to 1798' is a valid test statistic. I am quite sure that it has never been used in a peer-reviewed paper before. That does not mean that I can't perform valid statistical tests on that statistic. If, however, the incorrect test is applied, the results of the analysis will be flawed. Papers apply a (relatively small) standard repertoire of valid tests to a (potentially infinite) number of test statistics. The particular test statistic used in a paper may never have been used before and may never be used again; that does not address the validity of the analysis. In Blount's case, the test is the Monte Carlo analysis, which is also "widely-used by statisticians".

::::There are an infinite number of ways to reduce a data set to a single number. However, it’s foolish to think every method would be effective. I gave an example of a flawed test statistic in an earlier post [http://www.conservapedia.com/index.php?title=Talk%3ASignificance_of_E._Coli_Evolution_Experiments&diff=635070&oldid=634987]. Another example of a flawed test statistic is the one used in the paper because it does not always detect deviations from the null hypothesis (see: [[Significance of E. Coli Evolution Experiments#Test Statistics]]).

::::Test statistics are typically derived. The likelihood ratio test is a common method used to derive them. The chi-square test for independence is an approximation to the LRT. Where is the derivation saying that mean mutation generation is an appropriate test statistic for this problem? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::We still haven't touched on the issue of the categories not being independent, which by itself is sufficient to invalidate the chi-squared technique. I'm new to this site, so I'm unsure as to the etiquette of making changes to the articles of another person - but the article here should at the very least mention that the chi-square test is being used here in a manner that violates its underlying assumptions in at least two fundamental ways, and the results are therefore suspect.--[[User:ElyM|ElyM]] 17:34, 12 March 2009 (EDT)

:::::When generating random realizations of experiment outcomes, the authors assumed that the total number of mutants was fixed. Thus the paper assumed the numbers of mutants per generation are statistically dependent. Does this seem like a realistic model, or do you think that if the experiments were recreated that the total number of mutants could vary? For example, if experiment one were recreated, would the total number of mutants always be exactly four? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::: It looks to me as though SJohnson has misinterpreted the application of the chi-squared test in quite a fundamental way. His/her analysis of Blount's data are therefore close to meaningless, regardless of whether the test used by Blount is appropriate or not. In my opinion, the entire page should therefore be deleted. [[User:FredFerguson|FredFerguson]] 08:18, 13 March 2009 (EDT)

::::: "Fred", perhaps you mistakenly think this is Wikipedia, where [[censorship]] and deletion of pages for ideological reasons are common. Not here.--[[User:Aschlafly|Andy Schlafly]] 10:23, 14 March 2009 (EDT)

:::::: Umm... I'm suggesting deletion for mathematical reasons, not ideological reasons. Using an argument filled with mathematical errors to try to support your case only detracts from your credibility. [[User:FredFerguson|FredFerguson]] 10:38, 14 March 2009 (EDT)

:::::: Actually, I think correction is better than deletion. So that's what I've done. [[User:FredFerguson|FredFerguson]] 11:01, 14 March 2009 (EDT)

::::::: I find no credibility in your denial of having ideological reasons.--[[User:Aschlafly|Andy Schlafly]] 11:04, 14 March 2009 (EDT)

== Misinterpretation of test ==

SJohnson, Your analysis misinterprets the test. You say the null hypothesis is that this mutation cannot happen. They saw a mutation (4 mutations, in fact, in the data set you show) so the null hypothesis (as you state is) is disproved. That's perfectly straightforward.

I don't know what the "mean mutation generation" test is but you're doing when you apply a chi-squared test to this dataset is to test if the mutations are evenly distributed throughout the generations. Your test says they are, so there's no strong evidence to suppose that mutations are likely to occur in one generation rather than another in the series of tests. Blount's test says thay aren't, so it's more likely that the mutation will occur later in the series of tests. I can't tell which test is right without knowing more about the test that Blount used.

But that point (the foregoing paragraph) has no bearing at all on the null hypothesis, as you describe it. The mutation appeared, so that means the hypothesis that the mutation can't happen is disproved. Very simple. [[User:FredFerguson|FredFerguson]] 21:10, 8 March 2009 (EDT)

:I never said that “the null hypothesis is that this mutation cannot happen”. The chi-square test statistic I'm using wouldn’t be defined if the null hypothesis mutation rate was zero because the <math>E\left[n_{i,j}\right]</math> term in the denominator of the statistic (see above equation) would be zero.

:The test statistic from the paper is the average of the generation numbers of observed mutations. For experiment one this number is
::<math>
\frac{1}{4}\left(30500+31500+2\times32500\right)= 31750.
</math>
:The same number is shown in Table 2 of the paper. [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

:: SJohnson, the way you're calculating the chi-squared statistic implies that you're testing the null hypothesis of a constant mutation rate over time against an alternative hypothesis of a mutation rate which varies over time. [[User:FredFerguson|FredFerguson]] 11:02, 9 March 2009 (EDT)

As it currently stands, the article makes the following statement: "The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown." This misstates the null hypothesis of the paper, which is elaborated in the Introduction section of the paper, and repeated in the section '''Statistical Analysis of the Replay Experiments''':
<blockquote>
For each experiment, we compared the observed mean generation of those clones that yielded Cit+ variants to the mean expected under the null hypothesis that clones from all generations have equal likelihood. The null thus corresponds to the rare-mutation hypothesis laid out in the Introduction."
Block quote
</blockquote><ref>www.pnas.org/cgi/reprint/105/23/7899.pdf</ref>

The article also continues to describe 'mean mutation generation' as a ''test'' rather than a ''statistic'' to which the ''Monte Carlo test'' was applied.--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

==Unreferenced Claims==

I deleted the claim that mean mutation generation is an appropriate test statistic because no reference was produced that back that claim. No reference was provided to back the claim that the chi-square test p-values are always conservative, either. [[User:SJohnson|SJohnson]] 12:57, 14 March 2009 (EDT)

: The reference is Everitt. I'll check I put it in the right place. [[User:FredFerguson|FredFerguson]] 13:30, 14 March 2009 (EDT)

There was a typo in my edit summaries on the talk page and the main page. I meant to say "Removed unsupported claims" rather than "Removed supported claims". [[User:SJohnson|SJohnson]] 13:13, 14 March 2009 (EDT)

I do not believe that anyone has claimed that 'chi-square test p-values are ''always'' conservative'. The claim that has been made is that ''under certain circumstances'', namely low n and low individual cell values, the chi-square test is an invalid test; that under those circumstances the power of the test is low and it becomes impossible to reject the null hypothesis even when it is false. You may have missed the pertinent sections in my links above, so I will directly quote the relevant sections. All the quoted sections refer to chi-square testing in particular. Any bolding below is mine.

:This edit claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639379]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

<blockquote>
"Assumptions: 

Even though a nonparametric statistic does not require a normally distributed population, there still are some restrictions regarding its use. 

1. Representative sample (Random) 
2. The data must be in frequency form (nominal data) or greater. 
3. The individual observations must be independent of each other. 
4. '''Sample size must be adequate. In a 2 x 2 table, Chi Square should not be used if n is less than 20. In a larger table, no expected value should be less than 1, and not more than 20% of the variables can have expected values of less than 5'''. 
5. Distribution basis must be decided on before the data is collected. 
6. The sum of the observed frequencies must equal the sum of the expected frequencies."
</blockquote>
<ref>http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm
</ref>

<blockquote>
"Assumptions:
* Random sample data are assumed. As with all significance tests, if you have population data, then any table differences are real and therefore significant. If you have non-random sample data, significance cannot be established, though significance tests are nonetheless sometimes utilized as crude "rules of thumb" anyway.
* A sufficiently large sample size is assumed, as in all significance tests. '''Applying chi-square to small samples exposes the researcher to an unacceptable rate of Type II errors. There is no accepted cutoff. Some set the minimum sample size at 50, while others would allow as few as 20'''. Note chi-square must be calculated on actual count data, not substituting percentages, which would have the effect of pretending the sample size is 100.
* '''Adequate cell sizes are also assumed. Some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count'''. When this assumption is not met, Yates' correction is applied.
* Independence. Observations must be independent. The same observation can only appear in one cell. '''This means chi-square cannot be used to test correlated data (ex., before-after, matched pairs, panel data)'''.
* Similar distribution. Observations must have the same underlying distribution.
* Known distribution. The hypothesized distribution is specified in advance, so that the number of observations that are expected to appear each cell in the table can be calculated without reference to the observed values. Normally this expected value is the crossproduct of the row and column marginals divided by the sample size.
* Non-directional hypotheses are assumed. Chi-square tests the hypothesis that two variables are related only by chance. If a significant relationship is found, this is not equivalent to establishing the researcher's hypothesis that A causes B, or that B causes A.
* Finite values. Observations must be grouped in categories.
* Normal distribution of deviations (observed minus expected values) is assumed. Note chi-square is a nonparametric test in the sense that is does not assume the parameter of normal distribution for the data -- only for the deviations.
* Data level. No assumption is made about level of data. Nominal, ordinal, or interval data may be used with chi-square tests."
</blockquote>
<ref>http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm</ref>

<blockquote>
"Assumptions: 
-None of the expected values may be less than 1 
-No more than 20% of the expected values may be less than 5"</blockquote> <ref>http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html</ref>

<blockquote>
"When performing a chi-square test, your data must satisfy important assumptions. Although these assumptions may be stated differently in different textbooks, they generally assert that: 
1)The sample must be randomly drawn from the population 
'''2)The sample size, n, must be large enough so that the expected cell count in each cell is greater than or equal to 5.''' 
Both assumptions must be met in the process of collecting your data, and violations of the second assumption will appear in the Minitab output when you run the analysis. 
... 
'''You may wonder why the second assumption is necessary for performing the chi-square test. The second assumption arises because the distribution of counts under the null hypothesis is multinomial, and the normal distribution can be used to approximate the multinomial distribution if the sample size is sufficiently large and the probability parameters aren't too small. It can be shown via the Central Limit Theorem that the multinomial distribution converges to the normal distribution as the sample size approaches infinity; however, there is no easy way to show mathematically how and when the convergence fails.'''"
</blockquote>
<ref>http://www.minitab.com/support/docs/Answers/Chi-Square%20Test%20Assumptions.pdf</ref>

<blockquote>
"'''The chi-square test is simpler to calculate but yields only an approximate P value. ... You should definitely avoid the chi-square test when the numbers in the contingency table are very small (any number less than about six)'''."
</blockquote> <ref>http://www.graphpad.com/www/Book/Choose.htm</ref>

<blockquote>
"The most important things to remember to get a valid χ2 test are that the expected values are not too small in any bin (certainly 5 or more), and that the degrees of freedom are properly evaluated. '''Unless you have a very large amount of data, the test is not very sensitive and errs on the side of safety. If you get a significant result, however, it is not likely to be wrong.'''"
</blockquote> <ref>http://mysite.du.edu/~jcalvert/econ/chisquar.htm</ref>

<blockquote>
"The critical assumptions of the chi-square test for k independent samples are similar to those for the chi-square test for two independent samples. ... 
'''4. No more than 20% of the cells may have expected frequencies of less than 5, and no cell should have an expected frequency of less than 1. 
The rule given in Assumption 4 is particularly important for a contingency table that is larger than 2X2'''"
</blockquote> <ref>http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1
</ref>

<blockquote>
"Special problems with small expected cell frequencies for the chi-square test: 
The chi-square test involves using the chi-square distribution to approximate the underlying exact distribution. The approximation becomes better as the expected cell frequencies grow larger, and '''may be inappropriate for tables with very small expected cell frequencies.''' 
'''For tables with expected cell frequencies less than 5, the chi-square approximation may not be reliable. A standard (and conservative) rule of thumb (due to Cochran) is to avoid using the chi-square test for tables with expected cell frequencies less than 1, or when more than 20% of the table cells have expected cell frequencies less than 5.''' 
Another rule of thumb (due to Roscoe and Byars) is that the average expected cell frequency should be at least 1 when the expected cell frequencies are close to equal, and 2 when they are not. (If the chosen significance level is 0.01 instead of 0.05, then double these numbers.) 
Koehler and Larntz suggest that if the total number of observations is at least 10, the number categories is at least 3, and the square of the total number of observations is at least 10 times the number of categories, then the chi-square approximation should be reasonable. 
Care should be taken when cell categories are combined (collapsed together) to fix problems of small expected cell frequencies. Collapsing can destroy evidence of non-independence, so a failure to reject the null hypothesis for the collapsed table does not rule out the possibility of non-independence in the original table. 
'''As with most statistical tests, the power of the chi-square test increases with a larger number of observations. If there are too few observations, it may be impossible to reject the null hypothesis even if it is false.'''"
</blockquote> <ref>http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html</ref>--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

: Thanks for this really excellent contribution, ElyM. The only thing I'd like to add is in relation to your initial statement, "I do not believe that anyone has claimed that 'chi-square test p-values are always conservative'". The question of whether a test is conservative in a particular situation is probabilistic. One can determine whether a test is likely to generate a p-value which is too high in a particular situation (e.g. for a chi-squared test, when there are lots of small expected values) but one needs an exact test (such as an appropriate Monte Carlo randomisation test) to determine whether the p-value in any ''particular'' test is in fact excessively high.

::This edit also claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639373]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

: I hope careful reading of your very clear description will put SJohnson's mind at rest on this subject. [[User:FredFerguson|FredFerguson]] 18:11, 14 March 2009 (EDT)

ElyM, you've provided nothing to address the basic flaw that "The paper incorrectly applied a Monte Carlo resampling test to exclude the null hypothesis for rarely occurring events." See [[Flaws in Lenski Study]]. Also, do not impose your view on the content page until after SJohnson has had an opportunity to respond to your posting. As to "Fred", his put-downs are getting tiresome and I'm going to review his edit pattern now to see if he's been contributing anything of value to this site.--[[User:Aschlafly|Andy Schlafly]] 14:06, 15 March 2009 (EDT)

::Mr. Schlafly, per your request I have not added anything to the content page as SJohnson has not yet responded to my posts. Since all of my comments have been in regards to SJohnson's use of the chi-square test in this particular article, I'm not sure why you expect me to address Blout's use of Monte Carlo - that issue seems to be addressed on the [[Flaws in Lenski Study]] page. SJohnson has added a reformulation of the chi-square test for two possible outcomes, and stated that the chi-square test is at a minimum when all success probabilities are equal. He then extrapolates from this to claim that the chi-square test is an effective test for the data from Blount.

::The reformulation of the equations for two possible outcomes does not address the underlying problem that the chi-square test has universally accepted parameters outside of which it is considered an invalid test; I have provided references for these parameters and shown that the data from Blount lies outside them. None of the expected cells in SJohnson's analysis have values above one, and the total n is four. SJohnson's own reference states that the application of the chi-square test in this circumstance is a "violation of good statistical practice". Analogously, combining F=ma and t=(vf-vi)/a into t=(vf-vi)m/F and showing that t is a minimum when m approaches zero does not address the fact that those Newtonian equations do not apply as velocities approach the speed of light. The legitimacy of Blount's arguments cannot be determined by the application of illegitimate counterarguments. If SJohnson or others can point to references from the statistical literature that show that Blount has made methodological errors - as I have been able to do with SJohnson's chi-square analysis - I would welcome their input, and no doubt Conservapedia's other readers would as well, and this page would be greatly improved.

::I have not seen a rebuttal from SJohnson in the four days since my last post, although he has added new material to the content page since then. In light of this, I would appreciate some guidelines as to when it is appropriate for me to add my information and references to the content page. I can add citations from the primary mathematical literature if necessary, but in general I find that these are less helpful as they are not easily accessible by readers without access to academic libraries.--[[User:ElyM|ElyM]] 18:07, 18 March 2009 (EDT)

== References ==
{{reflist}}

Talk:Significance of E. Coli Evolution Experiments

2009-03-19T01:57:44Z

Argon:

SJohnson, your assessment, while good in the utilization of the chi-squared test is unfortunately incorrect. The Monte Carlo resampling gives a more accurate p-value than the chi-squared. You may research the literature (i.e. publications in statistical mathematics, many pubs actualy compare Monte Carlo vs Chi Squared) to discover that this method is commonly used in advance statistical work and how it is more accurate than the chi-squared test.--[[User:Able806|Able806]] 17:00, 4 March 2009 (EST)

:It doesn’t make sense to compare the chi-square test, which is a specific statistical hypothesis test, to Monte Carlo methods, which can be used for anything from fluid motion modeling to p-value computations. You can use Monte Carlo methods to compute the p-values of the chi-square test!

:Monte Carlo methods involve the generation of random realizations. Your broad claim the Monte Carlo methods are “more accurate” than the chi-square test is obviously incorrect because the accuracy of Monte Carlo methods always depends on the number of random realizations generated. When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous.

:Which publications compare Monte Carlo to chi-square and show that the former is more accurate? Could you provide specific examples? Thanks. [[User:SJohnson|SJohnson]] 18:50, 4 March 2009 (EST)

:In furtherance of SJohnson's remarks with respect to rarely occurring events, the use of the basic Monte Carlo method is plainly incorrect for modeling a rarely occurring event, as the Lenski paper did. This has long been pointed out in [[Flaws in Richard Lenski Study]]. I know [[evolutionists]] will never admit a flaw in anything promoting their pet theory, but this (and other) flaws in that paper is undeniable.

:Watch how evolutionists defended obvious errors in the Lenski paper, and then realize why the [[Piltdown Man]] fraud was taught for 40 years without evolutionists admitting it was a hoax.--[[User:Aschlafly|Andy Schlafly]] 09:55, 5 March 2009 (EST)

::Andy, how exactly is the Monte Carlo method incorrect to use in this case? I have seen it used in publications with much smaller datasets.--[[User:Able806|Able806]] 10:29, 5 March 2009 (EST)

:::Able806, I'm interested in looking at the publications you mentioned that use Monte Carlo methods to analyze small data sets. Could you provide some examples? Thanks. [[User:SJohnson|SJohnson]] 16:41, 5 March 2009 (EST)

::::SJohnson, here are two papers, [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6WH8-45RFJ1J-19&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=1ad95954654bb97b17e474ce6b469f6e 1] and [http://cat.inist.fr/?aModele=afficheN&cpsidt=787963 2]. Most are in chemistry and genetics where you find the observed to be much smaller and have to use the MCM. You can search on the subject as well and find that how Lenski performed the test is the standard for microbiological genetic analysis.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

:::::Those papers have nothing to do with hypothesis testing. One is an archeology paper. To be blunt, it seems like you’re just doing internet searches on “Monte Carlo” to find these links. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

::::::SJohnson, actually they do, did you read the papers? If so you would see how they used the MCM for their data analysis of small data sets, which indeed was hypothesis testing and answers you inquiry about publications that use MCM for small data set analysis. If you wish I can try to track down some actual mathematical publications, however, I am not as familiar with mathematical journals as I am with science/medical journals (not knowing which mathematical journals are acceptable). I am assuming that you have a background in math and possibly access to mathematical journals, therefore if you know the reputable ones I can do the leg work.
::::::I believe the thing that needs to be looked at is there truly a problem with the choice of test and if so what is an alternative. Bayesian might be an option but seems to be difficult to employ for this situation.--[[User:Able806|Able806]] 12:36, 12 March 2009 (EDT)

:::Able806, you still seem to miss the point about how inappropriate the Monte Carlo method (as used in the Lenski paper) is for evaluating rarely occurring events. You need to open your mind to be productive. If you simply cling to a view that Lenski (who I don't think has any meaningful education in statistics) must somehow be right, then you're not going to make any progress in understanding the flaws.--[[User:Aschlafly|Andy Schlafly]] 17:07, 5 March 2009 (EST)

::::Andy, you still have not answered what you find inappropriate about his use of the Monte Carlo method? I am a reasonable person and with evidence I do have an open mind. I provided examples last week, with a working model, showing that Monte Carlo is better than the chi-square in this case. I have also shown where the Chi-Square was inappropriate due to the occurrence size as well. So if you have any evidence that Monte Carlo should not be used in the way that Lenski used please let it be shown.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

Sjohnson, I believe you just proved my point. In the literature of mean and covariance structure analysis, non-central chi-square distribution is commonly used to describe the behavior of the likelihood ratio statistic under alternative hypothesis; it is widely believed that the non-central chi-square distribution is justified by statistical theory. Actually, when the null hypothesis is not trivially violated, the non-central chi-square distribution cannot describe the LR statistic well even when data are normally distributed and the sample size is large. Monte Carlo results compare the strength of the normal distribution against that of the non-central chi-square distribution. In an association analysis comparing cases and controls with respect to allele frequencies at a highly polymorphic locus, a potential problem is that the conventional chi-squared test may not be valid for a large, sparse contingency table. Reliance on statistics with known asymptotic distribution is unnecessary, as Monte Carlo simulations can be performed to estimate the significance level of the test statistic.

Here is a [http://faculty.vassar.edu/lowry/chi_beta.html link] to a great page the provides an interactive example as to why the Chi Squared test would provide poor results compared to the Monte Carlo in relation to the Lenski data workup.

Something you may have overlooked was that the data set is actually too small to use the chi square method correctly. It is often accepted that is any of the analyzed data falls under 10 for a particular cell of the data set then the Yates correction needs to be applied; unfortunately the Yates correction can over correct thus skewing the p-value. Lenksi seemed to understand this by supporting his Monte Carlo p-value results with the Fisher z-transformation p-value.

I hope this helps.--[[User:Able806|Able806]] 10:27, 5 March 2009 (EST)

:I’m still waiting to hear which literature says that “Monte Carlo resampling” is “more accurate than the chi-squared test”. The page mentioned above [http://faculty.vassar.edu/lowry/chi_beta.html] is a discussion of why statisticians “fail to reject the null” rather than “accepting the null” when the p-value is above 0.05 or so. The page says nothing about superiority of Monte Carlo methods. Why were alternate hypothesis distributions mentioned? Only the null hypothesis distribution is used to calculate a p-value. Yates’s correction is for 2x2 contingency tables [http://en.wikipedia.org/wiki/Yates%27_correction_for_continuity]. It doesn’t apply in this case. Finally, what the heck do “covariance structure analysis” and “allele frequencies at a highly polymorphic locus” have to do with this problem? [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::SJohnson, I am looking for this paper for you, I cited it for one of my past publications dealing with allele frequencies (I believe it came from the Duke Biostatistics group). To answer your question about allele frequencies, that is the issue at hand, more about the genetics than the math, but it is the item being studied. So you stated that Yates can not be used and statistics says the number of occurrences is too small to evaluate using the Chi-Squared test so what would you recommend instead of the Monte-Carlo Method?

:Regarding the "Fisher z-transformation p-value" from the paper, garbage in garbage out. If the p-values were bad to begin with, then why would a combination of them be meaningful? [[User:SJohnson|SJohnson]] 10:49, 9 March 2009 (EDT)

::You are assuming that p-values are wrong based on a test that is inappropriate in this case due to data limitations. Did you perform a z-transformation on the chi-squared for the three data groups?--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

::You asked about the “Fisher z-transformation p-value”. The z-transformation test and Fisher’s method are actually two different things (see Whitlock's 2005 paper - Ref. 49 in Blount et al.). But no, I haven’t tried either. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::There's a large literature on various kinds of Monte Carlo test, a very short summary of which is that they're inevitably more accurate than parametric tests (e.g. F, t, chi-squared, etc) because they don't make assumptions about the distribution of the data under the null hypothesis. See for example ''Introduction to the Bootstrap'' by B. Efron and R. Tibshirani and ''The Jack-knife, the Bootstrap and Other Resampling Plans'', also by Efron. They're certainly applicable to small datasets and their accuracy is really only limited by the number of samples you care to take. E.g. 1000 M-C samples would give you a pretty accurate idea about significance at the alpha<1% level (That book should answer SJohnson's questions of 18:50 on 4/3/09 and 16:38 on 5/3/09 about accuracy and Aschalfly's comment of 17:07 on 5/3/09 about appropriateness of Monte Carlo tests.) [[User:FredFerguson|FredFerguson]] 16:53, 11 March 2009 (EDT)

::::Your claim that Monte Carlo methods are “inevitably more accurate” than other tests is obviously wrong because the accuracy of MC methods always depends on the number of realizations used. You should have written <math>\alpha=1\%</math>, not <math>\alpha<1\%</math>. If 1,000 random realizations are generated, the number of realizations above the true <math>\alpha=1\%</math> level is binomial with mean 10 and variance about 10. Thus, the standard deviation of the MC estimate is >0.003. In this example, a Monte Carlo p-value could be off by 30% and still be within a standard deviation. Is that really “pretty accurate”?

::::Using one million MC realizations (as done in the paper) at the <math>\alpha=0.001</math> level means the standard deviation is about 3%. The paper reported a p-value of less than 0.001 (experiment two). It wouldn’t surprise me to find out that the experiment two p-value for the flawed test is off because only one million realizations were used. My original statement, “When p-values are small, Monte Carlo methods are notoriously inaccurate unless the number of realizations generated is enormous” is correct. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::::You're talking about miniscule differences in the accuracy of a test. 0.013 isn't very different from 0.007. In either case, it's very unlikely the experimenter would have obtained that result if the null hypothesis were true. If you're bothered about differences in P-values to the third decimals (which would make you unusual!), just run more MC realisations, that's all. Not really a problem. [[User:FredFerguson|FredFerguson]] 11:53, 12 March 2009 (EDT)

There’s still confusion about the difference between test statistics and Monte Carlo methods. Before you find a Monte Carlo estimate of a p-value, you need to select a test statistic to reduce the data set to a scalar. I am interested in hearing which test statistic you believe should be used in place of the chi-square test and why. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

----

Quick question for SJohnson: How many degrees of freedom did you choose when calculating the p-value? I'd like to know upon what condition you base that number. Thanks.--[[User:Argon|Argon]] 11:05, 5 March 2009 (EST)

:The degree of freedom for a contingency table is rows minus one times columns minus one. That is, <math> (r-1)(c-1) </math>. Here’s a pretty good tutorial I came across: [http://faculty.uncfsu.edu/dwallace/lesson%2020.pdf]. For the experiments from [http://www.pnas.org/content/105/23/7899.full.pdf], the DOFs are 11, 11, and 13. For experiment one, the chi-square test statistic is
::<math>
X^2
=\sum\limits_i\sum\limits_j
\frac{\left(n_{i,j}-E\left[n_{i,j}\right]\right)^2}
{E\left[n_{i,j}\right]}
</math>
::<math>
=\frac{\left(0-1/3\right)^2}{1/3}
+\frac{\left(6-17/3\right)^2}{17/3}
+\frac{\left(0-1/3\right)^2}{1/3}
+\ldots+
</math>
::<math>
+\frac{\left(2-1/3\right)^2}{1/3}
+\frac{\left(4-17/3\right)^2}{17/3}
</math>
::<math>
\approx
14.82
</math>
:where <math>n_{i,j}</math> is the observed value and <math>E\left[n_{i,j}\right]</math> is the expected null hypothesis value. So if you have MS Excel, another way to arrive at the p-value of 0.19 is to type “=CHIDIST(14.82,11)” into a cell. Cheers! [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

::OK, thanks for the info. From what I'd calculated and looked up in tables, the numbers seemed close to a df=11 for a chi-square of ~14. (Aside: With terms having 17/3 in the denominator in the figures above, were you using the test of independence? I was using Pearson's test for [http://en.wikipedia.org/wiki/Pearson%27s_chi-square_test#Test_for_fit_of_a_distribution fit of a distribution] which returns a chi-squared value of 14 and roughly matched the p-values you reported, assuming the df was 11).

::Also, the first sentence of the article reads: "Blount, Borland, and Lenski[1] claimed that a key evolutionary innovation was observed during a laboratory experiment. That claim is false." A small correction: There were several claims in the paper. The 'key evolutionary innovation' was acquiring the ability to utilize citrate as a food source. That claim was demonstrated multiple times. The claim, which pertains to this statistics discussion was that the Cit+ phenotype arose in a multi-step process, first requiring a rare, pre-adaptive mutation before additional mutation(s) lead to the subsequent development of citrate utilization.--[[User:Argon|Argon]] 20:46, 5 March 2009 (EST)

:::My biology-degreed wife assures me that mutation does not necessarily mean that evolution occurred. What the paper claimed is that evolution (a “key innovation”) occurred in the lab. The key innovation supposedly increased the mutation rate. In the experiments, the observed mutation rate increased after generation 31,000, but not enough to make a statistically significant claim that the rate is not constant. The analysis in the paper was similar to flipping a coin ten times, counting six heads and claiming that the coin must be biased against tails. In reality, there’s nothing surprising about a fair coin producing slightly more of one outcome than the other. Just like there's nothing surprising about there being slightly more mutations in later generations than early generations given the null hypothesis (constant mutation rate). [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

>>Insert later comment first<<
SJohnson, the paper's title is: "Historical contingency '''and the evolution of a key innovation''' in an experimental population of ''Escherichia coli''" As I mentioned earlier, the key innovation is the evolution of the Cit+ phenotype and not the timing or rate of its acquisition. And yes, it *is* evolution (call it microevolution, if you wish). Blount et al went on further to speculate how this evolutionary innovation arose and they proposed the historical contingency hypothesis in which 'pre-adaptive' mutations were required before the Cit+ phenotype developed. It is only this latter hypothesis that you are attempting to address with your chi-square analysis, not the fact that Cit+ mutants arose (which is the evolutionary innovation).--[[User:Argon|Argon]] 21:57, 18 March 2009 (EDT)

::::SJohnson, not to say anything about your wife, but has she had a 400 level molecular genetics course (most general biology degrees do not cover the detail unless they are specialized)? If so, she would have mentioned that if the mutation passes to the offspring and is selectively beneficial to the population then it is a step of evolution as along as the conditions continue through the sharing of the mutation with the population and the environment is such that reduces the growth rate of the non-transformed population. While not all mutations are signs that evolution occurred the mutations that pass to offspring and provide a benefit compared to other offspring are very strong indicators. In the case of this paper the population that evolved the cit+ was able to metabolize a chemical in their environment which allowed for an adaptation advantage compared to the non-transformed colonies.--[[User:Able806|Able806]] 10:19, 11 March 2009 (EDT)

----

Let’s go back to the beginning. There appears to be confusion about the difference between test statistics and methods for computing p-values. As is noted at the beginning of the page [http://www.conservapedia.com/Significance_of_E._Coli_Evolution_Experiments], the fundamental problem with the paper is that it used a flawed test statistic, not that it used Monte Carlo methods to find the p-value for that flawed statistic.

Every hypothesis test uses a test statistic to reduce the data to a single number. The p-value for the test statistic can be calculated analytically (as I’ve done for the chi-square test statistic) or by Monte Carlo methods. In the paper, Monte Carlo methods were used to compute the p-value of the “mutation generation” test statistic. The key problem with the analysis from the paper is that it doesn’t work to use a weighted average to test for variations in mutation rate. This is like trying to use the sample variance to test for an increase in the mean in Gaussian-distributed data. A statistic should be selected based on the null and alternate hypothesis distributions of the data. The chi-square test (unlike the weighted average from the paper) is a reasonable choice for data that mutates at a constant rate under the null hypothesis, but mutates at varying rates under the alternate hypothesis.

Able806, you made a good point about the contingency table cell frequencies being relatively low, but were wrong when you said ”the data set is actually too small to use the chi square method correctly”. In the low cell frequency case the chi-square test is still effective, but the null hypothesis distribution of the chi-square statistic starts to look less like the chi-square distribution. Thus, p-values calculated using the chi-square distribution may be a bit off. However, Monte Carlo p-values are always imperfect as well because it's impossible to generate an infinite number of random realizations. There are imperfections in p-values generated by analytic and Monte Carlo methods. However, low cell frequencies does not explain the >20x and >2.5x differences between chi-square p-values and p-values from the paper for experiments one and three. The reason for those huge differences was the use of the flawed test statistic (“mutation generation”) in the paper. [[User:SJohnson|SJohnson]] 16:38, 5 March 2009 (EST)

:SJohnson, the chi-squared test is a valuable statistical tool, but the limitations of the test must be acknowledged. The chi-squared test can only produce valid results if the assumptions that underly the test are not violated. As an analogy, Newtonian models of motion fail to produce accurate results as velocities approach the speed of light; under those circumstances one must switch to a theory that accounts for relativistic effects.

:It seems that you have simply dismissed the [http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm widely-acknowledged] [http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm fact] that the [http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html chi-squared test] is [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236 inappropriate] for use in [http://www.graphpad.com/www/Book/Choose.htm situations] where n in any cell is [http://mysite.du.edu/~jcalvert/econ/chisquar.htm less] less than a [http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1 threshold] [http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html number]. Different authors set different thresholds, but all are well above the numbers seen in your chi-squared analysis - even the most liberal guidelines advise against the chi-squared test when any expected cell frequency is less than one or more than 20% of the table cells are less than 5; others require that expected values in all cells must be more than 5. With smaller amounts of data, the test is insensitive and errs on the side of rejecting the hypothesis. If you attempt your chi-squared statistical analysis with a program that is more sophisticated than MS Excel (as I did), you get an error message indicating that the results are invalid due to low expected cell counts.

:That issue aside, there are other reasons that the chi-squared test is inappropriate here. As the links above point out, the categories tested must be truly independent; one example is that you can't use the chi-squared test to compare age and ability to kick a field goal by testing the same experimental group twice, one year apart; you have to test one group of age A and a different group of age B. In the case of the Blount paper, the categories are not independent. Even if there were adequate numbers to address the low-expected-frequency problem, this would make the chi-squared an invalid test in this case.

:There are other significant problems with the use of the chi-squared test in this circumstance, but they can wait until you address these first major problems.--[[User:ElyM|ElyM]] 12:18, 11 March 2009 (EDT)

::Wackerly et al. says in general it’s assumed that the cell frequencies are above five so that the chi-square statistic (under the null) is approximately chi-square distributed (see p. 703). That book does not say chi-square test results are invalid if frequencies are five or less. Your example of a chi-square test warning message (it said "warning" not "error" as you stated) in Minitab [http://www.minitab.com/support/answers/answer.aspx?log=0&id=2236] said “approximation probably invalid” referring to the chi-square distribution approximation to the chi-square test statistic’s distribution. Your example did not say “chi-square test invalid”. I agree that when cell frequencies are low, the chi-square test statistic’s distribution starts to deviate from the chi-square distribution. I maintain that this deviation is not enough to explain the >2.5x and >20x differences in the chi-square test p-values and the p-values from the paper.

::As the numerous links in your post proved, the chi-square test is widely-used by statisticians. Can you give examples of statisticians using mean mutation generation as a test statistic? Also, did your software agree with the chi-square test p-values I presented? Thanks. [[User:SJohnson|SJohnson]] 10:10, 12 March 2009 (EDT)

:::Thank you for giving page references for Wackerly; however it seems we have different editions, since page 703 in my copy (5th ed, 1996) does not deal with chi-squared issues at all. My copy does state the following, on page 622: "Although the mathematical proof is beyond the scope of this text, it can be shown that, when n is large [chi-squared] will possess approximately a chi-square probability distribution in repeated sampling." Then, on page 624: "Experience has shown that cell counts [n sub i] should not be too small in order that the chi-square distribution provide an accurate approximation to the distribution of [chi squared]. As a rule of thumb we require that all expected cell counts equal or exceed 5, although Cochran (1952) has noted that this value can be as low as 1 for some situations." Wackerly then goes on, in the problems sections, to describe the use of the chi-squared test as a "violation of good statistical practice" when "some expected counts [are] <5."

:::It seems that you are already aware that the [chi-square] statistic under the null is no longer chi-square distributed for small n; this is precisely why the test should not be used under those conditions. I can claim to be able to accelerate a 1-kg mass to 10 times the speed of light by applying 1 N of force for 95 years by using F=ma and t= (vf-vi)/a. Plugging the numbers into those equations will produce the same result every time, but the answer is illegitimate because those equations are only valid under certain assumptions, which are violated as velocities approach the speed of light. Similarly, having a statistical program calculate a chi-squared value given the Blount data will produce a number result, but since the assumptions of the test are violated the result is not legitimate. Yes, if I put the Blount data in SAS 9.2, I get the same numerical answer as you do, but I also get the following message: "WARNING: >89% of the cells have expected counts less than 5. Chi-square may not be a valid test." You may argue that that's a warning, not an error; that's a semantic distinction. The reason that the program says that it MAY not be valid is that the chi-squared test skews in the direction of being too conservative at low n values; the test has an acceptable rate of false positives but an unacceptably high rate of false negatives. Comparing the results of the Monte Carlo and chi-squared results in this case is like comparing the results of Newtonian and relativistic equations of motion: they can produce very different results from the same input data.

::::For a finite amount of data, the chi-square statistic is never chi-square distributed under the null. The p-values are always approximate regardless of cell frequencies. The approximation becomes more accurate as the amount of data increases, but I don’t believe that this inaccuracy will change p-values that are about 0.2 (for experiments 1 and 3) into statistically significant p-values. How much do you expect the p-values to change if an exact computation is used in place of the chi-square distribution approximation? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::Your last paragraph has a major non sequitur in it: yes, many statisticians use the chi-square test. As long as the assumptions of the test are not violated, it is a valuable tool. That has nothing to do with the validity of using mean mutation generation as a test statistic. 'Mean number of werewolf attacks in Mumbai in the week centered on the new moon, by month, from 1654 to 1798' is a valid test statistic. I am quite sure that it has never been used in a peer-reviewed paper before. That does not mean that I can't perform valid statistical tests on that statistic. If, however, the incorrect test is applied, the results of the analysis will be flawed. Papers apply a (relatively small) standard repertoire of valid tests to a (potentially infinite) number of test statistics. The particular test statistic used in a paper may never have been used before and may never be used again; that does not address the validity of the analysis. In Blount's case, the test is the Monte Carlo analysis, which is also "widely-used by statisticians".

::::There are an infinite number of ways to reduce a data set to a single number. However, it’s foolish to think every method would be effective. I gave an example of a flawed test statistic in an earlier post [http://www.conservapedia.com/index.php?title=Talk%3ASignificance_of_E._Coli_Evolution_Experiments&diff=635070&oldid=634987]. Another example of a flawed test statistic is the one used in the paper because it does not always detect deviations from the null hypothesis (see: [[Significance of E. Coli Evolution Experiments#Test Statistics]]).

::::Test statistics are typically derived. The likelihood ratio test is a common method used to derive them. The chi-square test for independence is an approximation to the LRT. Where is the derivation saying that mean mutation generation is an appropriate test statistic for this problem? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::We still haven't touched on the issue of the categories not being independent, which by itself is sufficient to invalidate the chi-squared technique. I'm new to this site, so I'm unsure as to the etiquette of making changes to the articles of another person - but the article here should at the very least mention that the chi-square test is being used here in a manner that violates its underlying assumptions in at least two fundamental ways, and the results are therefore suspect.--[[User:ElyM|ElyM]] 17:34, 12 March 2009 (EDT)

:::::When generating random realizations of experiment outcomes, the authors assumed that the total number of mutants was fixed. Thus the paper assumed the numbers of mutants per generation are statistically dependent. Does this seem like a realistic model, or do you think that if the experiments were recreated that the total number of mutants could vary? For example, if experiment one were recreated, would the total number of mutants always be exactly four? [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

:::: It looks to me as though SJohnson has misinterpreted the application of the chi-squared test in quite a fundamental way. His/her analysis of Blount's data are therefore close to meaningless, regardless of whether the test used by Blount is appropriate or not. In my opinion, the entire page should therefore be deleted. [[User:FredFerguson|FredFerguson]] 08:18, 13 March 2009 (EDT)

::::: "Fred", perhaps you mistakenly think this is Wikipedia, where [[censorship]] and deletion of pages for ideological reasons are common. Not here.--[[User:Aschlafly|Andy Schlafly]] 10:23, 14 March 2009 (EDT)

:::::: Umm... I'm suggesting deletion for mathematical reasons, not ideological reasons. Using an argument filled with mathematical errors to try to support your case only detracts from your credibility. [[User:FredFerguson|FredFerguson]] 10:38, 14 March 2009 (EDT)

:::::: Actually, I think correction is better than deletion. So that's what I've done. [[User:FredFerguson|FredFerguson]] 11:01, 14 March 2009 (EDT)

::::::: I find no credibility in your denial of having ideological reasons.--[[User:Aschlafly|Andy Schlafly]] 11:04, 14 March 2009 (EDT)

== Misinterpretation of test ==

SJohnson, Your analysis misinterprets the test. You say the null hypothesis is that this mutation cannot happen. They saw a mutation (4 mutations, in fact, in the data set you show) so the null hypothesis (as you state is) is disproved. That's perfectly straightforward.

I don't know what the "mean mutation generation" test is but you're doing when you apply a chi-squared test to this dataset is to test if the mutations are evenly distributed throughout the generations. Your test says they are, so there's no strong evidence to suppose that mutations are likely to occur in one generation rather than another in the series of tests. Blount's test says thay aren't, so it's more likely that the mutation will occur later in the series of tests. I can't tell which test is right without knowing more about the test that Blount used.

But that point (the foregoing paragraph) has no bearing at all on the null hypothesis, as you describe it. The mutation appeared, so that means the hypothesis that the mutation can't happen is disproved. Very simple. [[User:FredFerguson|FredFerguson]] 21:10, 8 March 2009 (EDT)

:I never said that “the null hypothesis is that this mutation cannot happen”. The chi-square test statistic I'm using wouldn’t be defined if the null hypothesis mutation rate was zero because the <math>E\left[n_{i,j}\right]</math> term in the denominator of the statistic (see above equation) would be zero.

:The test statistic from the paper is the average of the generation numbers of observed mutations. For experiment one this number is
::<math>
\frac{1}{4}\left(30500+31500+2\times32500\right)= 31750.
</math>
:The same number is shown in Table 2 of the paper. [[User:SJohnson|SJohnson]] 10:46, 9 March 2009 (EDT)

:: SJohnson, the way you're calculating the chi-squared statistic implies that you're testing the null hypothesis of a constant mutation rate over time against an alternative hypothesis of a mutation rate which varies over time. [[User:FredFerguson|FredFerguson]] 11:02, 9 March 2009 (EDT)

As it currently stands, the article makes the following statement: "The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown." This misstates the null hypothesis of the paper, which is elaborated in the Introduction section of the paper, and repeated in the section '''Statistical Analysis of the Replay Experiments''':
<blockquote>
For each experiment, we compared the observed mean generation of those clones that yielded Cit+ variants to the mean expected under the null hypothesis that clones from all generations have equal likelihood. The null thus corresponds to the rare-mutation hypothesis laid out in the Introduction."
Block quote
</blockquote><ref>www.pnas.org/cgi/reprint/105/23/7899.pdf</ref>

The article also continues to describe 'mean mutation generation' as a ''test'' rather than a ''statistic'' to which the ''Monte Carlo test'' was applied.--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

==Unreferenced Claims==

I deleted the claim that mean mutation generation is an appropriate test statistic because no reference was produced that back that claim. No reference was provided to back the claim that the chi-square test p-values are always conservative, either. [[User:SJohnson|SJohnson]] 12:57, 14 March 2009 (EDT)

: The reference is Everitt. I'll check I put it in the right place. [[User:FredFerguson|FredFerguson]] 13:30, 14 March 2009 (EDT)

There was a typo in my edit summaries on the talk page and the main page. I meant to say "Removed unsupported claims" rather than "Removed supported claims". [[User:SJohnson|SJohnson]] 13:13, 14 March 2009 (EDT)

I do not believe that anyone has claimed that 'chi-square test p-values are ''always'' conservative'. The claim that has been made is that ''under certain circumstances'', namely low n and low individual cell values, the chi-square test is an invalid test; that under those circumstances the power of the test is low and it becomes impossible to reject the null hypothesis even when it is false. You may have missed the pertinent sections in my links above, so I will directly quote the relevant sections. All the quoted sections refer to chi-square testing in particular. Any bolding below is mine.

:This edit claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639379]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

<blockquote>
"Assumptions: 

Even though a nonparametric statistic does not require a normally distributed population, there still are some restrictions regarding its use. 

1. Representative sample (Random) 
2. The data must be in frequency form (nominal data) or greater. 
3. The individual observations must be independent of each other. 
4. '''Sample size must be adequate. In a 2 x 2 table, Chi Square should not be used if n is less than 20. In a larger table, no expected value should be less than 1, and not more than 20% of the variables can have expected values of less than 5'''. 
5. Distribution basis must be decided on before the data is collected. 
6. The sum of the observed frequencies must equal the sum of the expected frequencies."
</blockquote>
<ref>http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm
</ref>

<blockquote>
"Assumptions:
* Random sample data are assumed. As with all significance tests, if you have population data, then any table differences are real and therefore significant. If you have non-random sample data, significance cannot be established, though significance tests are nonetheless sometimes utilized as crude "rules of thumb" anyway.
* A sufficiently large sample size is assumed, as in all significance tests. '''Applying chi-square to small samples exposes the researcher to an unacceptable rate of Type II errors. There is no accepted cutoff. Some set the minimum sample size at 50, while others would allow as few as 20'''. Note chi-square must be calculated on actual count data, not substituting percentages, which would have the effect of pretending the sample size is 100.
* '''Adequate cell sizes are also assumed. Some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count'''. When this assumption is not met, Yates' correction is applied.
* Independence. Observations must be independent. The same observation can only appear in one cell. '''This means chi-square cannot be used to test correlated data (ex., before-after, matched pairs, panel data)'''.
* Similar distribution. Observations must have the same underlying distribution.
* Known distribution. The hypothesized distribution is specified in advance, so that the number of observations that are expected to appear each cell in the table can be calculated without reference to the observed values. Normally this expected value is the crossproduct of the row and column marginals divided by the sample size.
* Non-directional hypotheses are assumed. Chi-square tests the hypothesis that two variables are related only by chance. If a significant relationship is found, this is not equivalent to establishing the researcher's hypothesis that A causes B, or that B causes A.
* Finite values. Observations must be grouped in categories.
* Normal distribution of deviations (observed minus expected values) is assumed. Note chi-square is a nonparametric test in the sense that is does not assume the parameter of normal distribution for the data -- only for the deviations.
* Data level. No assumption is made about level of data. Nominal, ordinal, or interval data may be used with chi-square tests."
</blockquote>
<ref>http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm</ref>

<blockquote>
"Assumptions: 
-None of the expected values may be less than 1 
-No more than 20% of the expected values may be less than 5"</blockquote> <ref>http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html</ref>

<blockquote>
"When performing a chi-square test, your data must satisfy important assumptions. Although these assumptions may be stated differently in different textbooks, they generally assert that: 
1)The sample must be randomly drawn from the population 
'''2)The sample size, n, must be large enough so that the expected cell count in each cell is greater than or equal to 5.''' 
Both assumptions must be met in the process of collecting your data, and violations of the second assumption will appear in the Minitab output when you run the analysis. 
... 
'''You may wonder why the second assumption is necessary for performing the chi-square test. The second assumption arises because the distribution of counts under the null hypothesis is multinomial, and the normal distribution can be used to approximate the multinomial distribution if the sample size is sufficiently large and the probability parameters aren't too small. It can be shown via the Central Limit Theorem that the multinomial distribution converges to the normal distribution as the sample size approaches infinity; however, there is no easy way to show mathematically how and when the convergence fails.'''"
</blockquote>
<ref>http://www.minitab.com/support/docs/Answers/Chi-Square%20Test%20Assumptions.pdf</ref>

<blockquote>
"'''The chi-square test is simpler to calculate but yields only an approximate P value. ... You should definitely avoid the chi-square test when the numbers in the contingency table are very small (any number less than about six)'''."
</blockquote> <ref>http://www.graphpad.com/www/Book/Choose.htm</ref>

<blockquote>
"The most important things to remember to get a valid χ2 test are that the expected values are not too small in any bin (certainly 5 or more), and that the degrees of freedom are properly evaluated. '''Unless you have a very large amount of data, the test is not very sensitive and errs on the side of safety. If you get a significant result, however, it is not likely to be wrong.'''"
</blockquote> <ref>http://mysite.du.edu/~jcalvert/econ/chisquar.htm</ref>

<blockquote>
"The critical assumptions of the chi-square test for k independent samples are similar to those for the chi-square test for two independent samples. ... 
'''4. No more than 20% of the cells may have expected frequencies of less than 5, and no cell should have an expected frequency of less than 1. 
The rule given in Assumption 4 is particularly important for a contingency table that is larger than 2X2'''"
</blockquote> <ref>http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1
</ref>

<blockquote>
"Special problems with small expected cell frequencies for the chi-square test: 
The chi-square test involves using the chi-square distribution to approximate the underlying exact distribution. The approximation becomes better as the expected cell frequencies grow larger, and '''may be inappropriate for tables with very small expected cell frequencies.''' 
'''For tables with expected cell frequencies less than 5, the chi-square approximation may not be reliable. A standard (and conservative) rule of thumb (due to Cochran) is to avoid using the chi-square test for tables with expected cell frequencies less than 1, or when more than 20% of the table cells have expected cell frequencies less than 5.''' 
Another rule of thumb (due to Roscoe and Byars) is that the average expected cell frequency should be at least 1 when the expected cell frequencies are close to equal, and 2 when they are not. (If the chosen significance level is 0.01 instead of 0.05, then double these numbers.) 
Koehler and Larntz suggest that if the total number of observations is at least 10, the number categories is at least 3, and the square of the total number of observations is at least 10 times the number of categories, then the chi-square approximation should be reasonable. 
Care should be taken when cell categories are combined (collapsed together) to fix problems of small expected cell frequencies. Collapsing can destroy evidence of non-independence, so a failure to reject the null hypothesis for the collapsed table does not rule out the possibility of non-independence in the original table. 
'''As with most statistical tests, the power of the chi-square test increases with a larger number of observations. If there are too few observations, it may be impossible to reject the null hypothesis even if it is false.'''"
</blockquote> <ref>http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html</ref>--[[User:ElyM|ElyM]] 17:24, 14 March 2009 (EDT)

: Thanks for this really excellent contribution, ElyM. The only thing I'd like to add is in relation to your initial statement, "I do not believe that anyone has claimed that 'chi-square test p-values are always conservative'". The question of whether a test is conservative in a particular situation is probabilistic. One can determine whether a test is likely to generate a p-value which is too high in a particular situation (e.g. for a chi-squared test, when there are lots of small expected values) but one needs an exact test (such as an appropriate Monte Carlo randomisation test) to determine whether the p-value in any ''particular'' test is in fact excessively high.

::This edit also claimed that chi-square test p-values are conservative, but didn't back that claim with a reference: [http://www.conservapedia.com/index.php?title=Significance_of_E._Coli_Evolution_Experiments&diff=next&oldid=639373]. [[User:SJohnson|SJohnson]] 20:49, 18 March 2009 (EDT)

: I hope careful reading of your very clear description will put SJohnson's mind at rest on this subject. [[User:FredFerguson|FredFerguson]] 18:11, 14 March 2009 (EDT)

ElyM, you've provided nothing to address the basic flaw that "The paper incorrectly applied a Monte Carlo resampling test to exclude the null hypothesis for rarely occurring events." See [[Flaws in Lenski Study]]. Also, do not impose your view on the content page until after SJohnson has had an opportunity to respond to your posting. As to "Fred", his put-downs are getting tiresome and I'm going to review his edit pattern now to see if he's been contributing anything of value to this site.--[[User:Aschlafly|Andy Schlafly]] 14:06, 15 March 2009 (EDT)

::Mr. Schlafly, per your request I have not added anything to the content page as SJohnson has not yet responded to my posts. Since all of my comments have been in regards to SJohnson's use of the chi-square test in this particular article, I'm not sure why you expect me to address Blout's use of Monte Carlo - that issue seems to be addressed on the [[Flaws in Lenski Study]] page. SJohnson has added a reformulation of the chi-square test for two possible outcomes, and stated that the chi-square test is at a minimum when all success probabilities are equal. He then extrapolates from this to claim that the chi-square test is an effective test for the data from Blount.

::The reformulation of the equations for two possible outcomes does not address the underlying problem that the chi-square test has universally accepted parameters outside of which it is considered an invalid test; I have provided references for these parameters and shown that the data from Blount lies outside them. None of the expected cells in SJohnson's analysis have values above one, and the total n is four. SJohnson's own reference states that the application of the chi-square test in this circumstance is a "violation of good statistical practice". Analogously, combining F=ma and t=(vf-vi)/a into t=(vf-vi)m/F and showing that t is a minimum when m approaches zero does not address the fact that those Newtonian equations do not apply as velocities approach the speed of light. The legitimacy of Blount's arguments cannot be determined by the application of illegitimate counterarguments. If SJohnson or others can point to references from the statistical literature that show that Blount has made methodological errors - as I have been able to do with SJohnson's chi-square analysis - I would welcome their input, and no doubt Conservapedia's other readers would as well, and this page would be greatly improved.

::I have not seen a rebuttal from SJohnson in the four days since my last post, although he has added new material to the content page since then. In light of this, I would appreciate some guidelines as to when it is appropriate for me to add my information and references to the content page. I can add citations from the primary mathematical literature if necessary, but in general I find that these are less helpful as they are not easily accessible by readers without access to academic libraries.--[[User:ElyM|ElyM]] 18:07, 18 March 2009 (EDT)

== References ==
{{reflist}}

World History Lecture Eight

2009-03-19T01:40:41Z

Argon: Living in plush quarters is not like relocation to derelict horse stables and tar paper shacks (ask a Japanese-American sometime)

[[World History Lecture One|1]]-[[World History Lecture Two|2]]-[[World History Lecture Three|3]]-[[World History Lecture Four|4]]-[[World History Lecture Five|5]]-[[World History Lecture Six|6]]-[[World History Lecture Seven|7]]-[[World History Lecture Eight|8]]-[[World History Lecture Nine|9]]-[[World History Lecture Ten|10]]-[[World History Lecture Eleven|11]]-[[World History Lecture Twelve|12]]-[[World History Lecture Thirteen|13]]-[[World History Lecture Fourteen|14]]

Our midterm exam will be next week. It will be multiple-choice, perhaps 30 minutes in length, and will cover lectures one through eight (that includes this one). Items and topics mentioned on the homework will be given greater priority in designing the questions.

This exam will be less detailed than prior exams. Try to understand the "big picture" of world history. Do not expect to be asked about specific dates, specific Roman emperors, or other detailed matters. Focus more on trends and broader concepts.

Remember and use the following test-taking tips for multiple-choice exams. You can improve your scores enormously by following and mastering these tips.

== Test-taking tips ==

'''Understand''': Understand the question before you try to answer it. Read the question twice if you have doubts. You want an answer that fits the question best, like a key fitting a lock.

'''Eliminate''': Eliminate the wrong answers before picking the right one. This can bump your score up another point or two.

'''First Impression''': Go with your first impression unless you have a good reason to change it. More often than not, your first impression will be right.

'''Stay on Track''': Stay on track during the test. Expect to miss a few, and do not waste too much time on a difficult question.

'''Try Different Approaches''': Try different approaches to a difficult question until you figure out the right answer. There are many ways to find the correct answer for a question. Don't give up too easily on figuring out a question.

'''Avoid Broad Language in Answers''': Broad or sweeping terms like "only", "never", "solely", "everyone", "always", "purely", "all" and "every" are often signs of '''incorrect''' answers because they are rarely true.

'''Try to Answer Every Question''': You can't increase your score by refusing to answer, just as you can't score a basket in basketball without tossing the ball towards the net. Trying is better than not trying. Try hard on every question, and if no points are deducted for wrong answers, be sure to answer every question. Even if you don't know the answer, try to make an educated guess.

Test-taking on multiple-choice exams is a skill that can be mastered with practice, determination and cleverness.

== Introduction ==

This lecture will be briefer than usual, with no homework questions, in order to give you time to prepare for the exam.

On September 11, 1683, Muslims made a second major attempt to conquer Europe (recall that Charles the Hammer turned back their first attempt). A massive Islamic army arrived at the Gates of Vienna. An alliance of Christian armies, led by the Polish King Jan III Sobieski, defeated the Muslim force.<ref>http://gatesofvienna.blogspot.com/2006/09/other-september-11th.html</ref> Some have observed that the "9/11" massacre in New York City was on the anniversary of this day.

Two other momentous events happened in the 1600s which would also affect the world forever. In 1607, Englishmen established a permanent settlement at Jamestown, Virginia. In 1611, the King James Bible was published with this introduction:<ref name="KJV">http://www.av1611.org/kjv/kjvhist.html</ref>

:THE HOLY BIBLE, Conteyning the Old Testament, and the New: Newly Translated out of the Originall tongues: & with the former Translations diligently compared and revised, by his Majesties Special Commandment. Appointed to be read in Churches. Imprinted at London by Robert Barker, Printer to the Kings most Excellent Majestie. ANNO DOM. 1611.

Some might say that the publication of the King James Bible was more significant than the settlement at Jamestown. This version of the Bible, which became known as the "Authorized Version," was a spectacular work that transformed and elevated the entire English language to new heights. Even a Catholic scholar expressed the highest praise for this Protestant work, as all Christians recognized it to be an absolute masterpiece:<ref name="KJV"/>

:The highest eulogiums have been made on the translation of James the First, both by our own writers and by foreigners. And, indeed, if accuracy, fidelity, and the strictest attention to the letter of the text, be supposed to constitute the qualities of an excellent version, this of all versions, must, in general, be accounted the most excellent. Every sentence, every work, every syllable, every letter and point, seem to have been weighed with the nicest exactitude; and expressed, either in the text, or margin, with the greatest precision.

To this day many feel that the King James Bible is the finest English translation available, despite the antiquity of some of its terminology. It is free from modern translation biases that hide the existence and wrath of Hell, for example. But for the purposes of this course, we are interested in how this was a defining moment in the development of the English language, and how the language has grown since then.

The King James Bible established a framework of knowledge of the English language, which began to develop rapidly afterward. English is unique in its ability to develop new words, at a rate of about 1000 new words per year.<ref>See [[Essay:Best New Conservative Words]].</ref> This enables English to constantly improve its ability to express concepts. English easily incorporates and adapts words from other languages, such as "detente" (French for a lessening in tensions between nations) and "tsunami" (Japanese for a massive tidal wave).

English has the smallest alphabet of widely used languages;<ref>Some describe the Hebrew alphabet, without vowels, as having only 22 characters, but modern Hebrew (spoken by a few million people) has more characters than English. See http://www.omniglot.com/writing/hebrew.htm . The obscure [[Elder Futhark]] runic writing has an alphabet of 24.</ref> this compact simplicity gives English an advantage, particularly in using computers. For example, modern English lacks accented letters (letters bearing a diacritic mark such as the "e" in "noël"), hybrid letters (such as the archaic digraph ligature æ used in British English in words such as encyclopædia or the ampersand character - a contraction of "et"), and pictograph characters.

It helps that English uses a writing system that has some relation between sounds and symbols as opposed to a purely ideographic system. This also applies to all other languages using alphabets, abjads, syllabaries, and so on - such as ancient Greek and Latin (though English developed from German, not from Greek or Latin).

English features easy interchangeability of nouns, verbs, and adjectives, without much variance in form for pronouns and verbs. That promotes easy communication through brief, cryptic messages if necessary, the style preferred by electronic media. English enjoys a powerful pipe-like quality, such that one phrase can be cut and pasted to another phrase with ease. Computer-based cutting and pasting text works more efficiently in English than in many other languages, such as those using pictorial characters. This gives English an advantage in the internet medium, and about 80% of the internet uses English. English is now the second most widely spoken language in the world, with only Chinese dialects spoken by more people. English is overwhelmingly the second language of choice for non-English-speaking people.<ref>See Barbara Wallraff, “What Global Language?” 286 Atlantic Monthly No. 5, at 52 (Nov. 2000) (noting, ''inter alia'', that “English is the working language of the Asian trade group ASEAN” and is also “the official language of the European Central Bank”).</ref>

==The Age of Exploration==

The “Age of Exploration” consisted of Christians from Europe searching the rest of our globe to spread the faith and seek wealth, particularly between 1450 and 1650. This began the “Global Age” in world history, which continues to this day.

As Europe was emerging from the Middle Ages and experiencing the Renaissance, people had the curiosity and the courage to look beyond their own continent to others. Scientific discoveries and technological improvements made it easier to travel across the vast oceans. Christians sought to spread their faith far and wide, and merchants sought new goods and business opportunities in both the East and West. North and South America awaited visits by the Europeans.
[[Image:Caravel.jpg|right|thumb|A caravel]]
The Portuguese were the first great explorers. They developed the “caravel”, a small, lightweight ship with three lateen-sailed masts that were faster and more maneuverable in shallow water than previous designs (see right). They could also hold a fair amount of spices, and the more cargo that a ship could hold, the greater its profitability would be. European demand for exotic Indian goods, especially spices, was strong, and this drove explorers to discover new routes to India different from the Eastern Mediterranean trade controlled by the Ottoman Muslims.

The discovery of new western routes accidentally led to the discovery of lands in the New World (Western Hemisphere). Portugal and Spain then emerged as the two major players in what was to become an intense European competition for exploration, conversion and colonization.

===Portuguese Exploration===

The Portuguese initiated the age of exploration in Europe as early as about 1419, when Prince Henry the Navigator established his School of Navigation and made it his life’s work to explore the West African coast and reach the Indian Ocean. His ships reached as far as Sierra Leone, but he failed to reach India. Another explorer named Bartholomew Diaz likewise failed to reach India, but he did manage to sail around the tip of Africa (the Cape of Good Hope) in 1487 and thereby establish the route to India.

In 1492, with funding from Spain (see below), Christopher Columbus embarked on his famous voyage with the dream of reaching India, and he failed in that goal also. It was Vasco da Gama who finally reached India by sailing eastward around Africa and up the East African coast between 1497 and 1499, with the help of the powerful monsoon winds. Shortly thereafter, regular trade was established with India under the Portuguese King Manuel, and conflict began with the Muslims over dominance of Indian Ocean trade.

Later, with funding and men from Spain (see below), the Portuguese navigator Ferdinand Magellan became the first to lead an expedition around the globe.

 
===Spanish Exploration===

The Spanish entered the scene in the year 1492 — during the reign of King Ferdinand and Queen Isabella, who were working hard to eradicate Islam and establish Christianity as the only religion in Spain. Known as the “Reconquista” (or Reconquest) of Spain, it ultimately succeeded in the reconquest of Granada, the last Muslim city in Spain. As explained in the prior Lecture, Queen Isabella ordered all Muslims and Jewish residents to convert or leave Spain. Initially, this resulted in an economic and academic setback for Spain, as Jewish residents had comprised a large number of scholars and merchants.

It was during this time that a young Italian by the name of Christopher Columbus first approached the Portuguese King John II to ask him for financial support. Columbus, like others from the Italian city of Genoa, was an expert mariner, having started as early as the age of 14. He had studied the works of Ptolemy on his own; Columbus did not attend formal school, just as homeschoolers do not today. Columbus at all times was a devout Christian, writing frequently about his faith in his diary. His desire to sail to India was motivated in large part by his attempt to spread the Christian faith. He repeatedly cited a desire to convert pagans to Christianity when he sought support for his voyage. Late in life, even when treated as a prisoner, he sought to devote all his money and efforts towards winning back Jesus Christ’s Holy Sepulchre (burial cave) from Muslims in the Holy Land.

King John II of Portugal denied Columbus’s request to fund his voyage across the Atlantic. Indeed, Columbus was ridiculed for seeking to reach India by sailing west rather than east.

The funding for Columbus came from Spain instead. Though Columbus was Italian, it was Spain that financed his voyages as the strain of fighting the Muslims subsided. Isabella and Ferdinand felt secure in granting the Genoese upstart Christopher Columbus his long-awaited funds. Beginning on August 3, 1492, Columbus embarked on four voyages, becoming perhaps the first European to discover the New World, although at first he believed he had reached India (hence his name for the inhabitants of the New World: “Indians”). The discoveries by Columbus included the Bahamas, Cuba, Haiti, San Salvador, Puerto Rico, Jamaica, Trinidad and Honduras. Columbus, the master mariner that he was, subsequently sailed back to Europe and then again to the Caribbean, and was able to find the exact same locations on his return voyage. Columbus’ astounding success presented an increasing challenge to Spain’s Portuguese rivals.

The pope finally resolved this rivalry between Spain and Portugal in the Treaty of Tordesillas in 1494, which established an 1100-mile long “Line of Demarcation,” which was 50 degrees longitude (north-to-south) in the Western Hemisphere. This division of world territories gave Portugal trading rights in India, China, the East Indies, East Brazil and the Spanish Americas. Spain was given control of the remaining, vast majority of the Americas. Ferdinand Magellan then led the first successful voyage to circumnavigate the globe in 1519. Although Magellan was killed by Philippine natives before completing the last leg of the journey, he was the first to sail around the tip of South America to enter the Pacific Ocean. Only 18 of Magellan’s approximately 250 sailors made it back to Spain three years later, in 1522.

As a result of Magellan’s journey, Spain gained control of many South American territories, including Peru. Spain also obtained control over the Philippines in the Far East. Spanish warrior Hernando Cortes led the conquest of Mexico, with the initial help of the Aztecs, many of whom believed Cortes was their god Quetzalcoatl (who, conveniently for the Spanish, was expected to appear that very year). Cortes captured the Aztec ruler Moctezuma (popularly known as “Montezuma”) and conquered their empire. Cortes was also aided by an Aztec woman named Malinche, who served as a guide and translator. Cortes burned the Aztec capital Tenochtitlan and built Mexico City in its place.

Francisco Pizzaro led the conquest of Peru, eventually killing their emperor, Atahualpa. The ancient Peruvian city of Cuzco was completely destroyed and replaced with the city of Lima. While in Peru, Pizzaro also discovered the greatest silver mine in the Americas. The vast majority of silver obtained was traded with China, whose economy was based on silver. Both Cortes’ and Pizzaro’s conquests were facilitated by the use of horses and superior weapons, and the South Americans were also devastated by European diseases like smallpox. The native population had no immunity to such disease, and some historians claim that the Spanish intentionally spread the deadly disease by giving the natives blankets that carried the germ. There is one defect to that anti-Christian theory: the germ theory was not discovered until the 19th century! No one knew that diseases were spread by germs in the 1500s, so the story is pure fiction.

Using the Spanish system of “encomiendas”, many of the South American natives became slaves to their Spanish conquerors. A Dominican monk and missionary, [[Bartholomew de Las Casas]], sent sensational descriptions to the Spanish government of the supposed atrocities committed against the “patient, meek and peaceful” natives by the Spanish conquistadors, whom he described as “cruel tigers, wolves and lions.” Although some of what Las Casas said was true, historical research has proved most of it to be false and exaggerated. However dishonest his methods may have been, Las Casas’ reports eventually led to the king of Spain issuing “New Laws,” by which exploitation of the natives was ended by the replacement of the old “encomiendas” with the new “repartimientos”. Both systems are now criticized as having been unfair. The system of “encomiendas” was based on royal grants of land, while the system of “repartimientos” was based on gifts of Indian labor. Either way, the Indians ended up working for the Spanish, but it was typical throughout history for conquerors to put the conquered people to work, as in "to the victors go the spoils."

===Exploration by Other Nations, and the Seven Years War===

Other European nations participated in exploration as well. King Henry VII of England sent John Cabot, a resident of London born in Genoa (like Columbus), to explore across the Northern Atlantic Ocean. Cabot discovered Newfoundland and the New England coast in 1497 and 1498, respectively, and the land became the property of England. From France, Jacques Cartier discovered Canada and the Mississippi Basin, including Louisiana in 1534 and 1541. These colonies were initially less significant than those in South America, and there was a lack of initial interest in North America because it did not have the gold and silver of South America. Whereas intermarriage and cultural blending occurred heavily among the Spanish and the Indians in South America, it barely existed between the English or French and the Indians in North America.

Of course, eventually the interest and development in North America increased and surpassed that of South America. Commercial competition between European nations culminated in the Seven Years’ War (1756-1763) — a conflict brought to America in the form of the French and Indian War. This War marked the end of French (and Spanish) influence in North America, and signaled the rise of Britain as the primary world superpower for the next 150 years.

The Seven Years War in Europe, of which the French and Indian War was a small part in North America, consisted of 20 major battles and ended up taking the lives of perhaps a million people. Britain and France fought in both Europe and North America; Spain and Portugal fought in South America. The war reached every inhabited continent except Australia, which had been discovered in only the 1600s and was not yet well-settled by Europeans. France, Russia, Sweden, Poland and Austria were allied against Britain and Prussia in the struggle for control of Silesia, a region of Eastern Europe that is mostly part of Poland today.

Britain won the war, partly due to the genius of William Pitt, after whom Pittsburgh is named. Britain gained both Canada and India as a result of the Seven Years War, but lost the American colonies two decades later in the Revolutionary War. Pitt himself had left power before the war concluded (though he returned to power later); he opposed the peace treaty on the grounds that it did not give enough to Britain. France was humiliated by this war, which may have contributed to the French Revolution a few decades later.

===The Slave Trade===

In what is known as the “Columbian Exchange,” plants and animals were brought from America to Europe and vice-versa. Columbus brought from Europe to America wheat, melons, onions, grapes, sugar cane and horses. Horses proved to be influential not only in the European conquest of native peoples but also in the natives’ abilities to avoid white settlers, particularly among the Plains Indians in North America.

America had new fruits and vegetables that were then exported to Europe and Africa. America introduced the white potato to Europe and the manioc or cassava to Africa. Chili peppers, pumpkins, squash and peanuts were also brought back from America and became popular in countries such as India. These new foods helped increase population, and in Africa the population growth almost canceled out population losses caused by the slave trade.

Slavery, though not considered part of the Columbian Exchange, became a hurtful form of trade. In 1455, a “papal bull” (formal letter by the pope) justified a “right” of Christian nations to enslave any non-Christians in the name of exploration. The Spanish had already been enslaving South American natives on a limited basis, but with the rise of sugar plantations the need for a larger slave force arose. Millions of African slaves were brought by the Spanish and Portuguese to Mexico, Peru, the Caribbean and especially Brazil. The growth of sugar — which had first been introduced to Europe when the Muslims ruled Spain — was exploding in popularity throughout the entire western world. Soon France, the Netherlands and Great Britain were also establishing profitable sugar plantations in the new world. The plantation system began in Brazil, where rich white plantation owners were the highest rung in the social hierarchy and the enslaved Africans were at the bottom. Obviously life on a sugar plantation was very hard work for a slave.

[[Slavery]] was widespread within Africa itself, and the richest in Africa were not those owning the most land, but those who owned the most slaves. In the Sahara Desert, slaves worked in caravans and were used in gold and salt mining. Slaves were usually prisoners of war from other areas of Africa, or debtors, or enemies of the king, but many women outside of those three categories were also enslaved in African societies. Polygamy — the practice of having more than one wife — was common in Africa, as was the existence of harems, from which African women were often sold to join Arabian or Middle Eastern harems.

The trading of slaves with other countries was encouraged in Africa, and was considered an important component of the African economy. Slave trade across the Atlantic (the Trans-Atlantic slave trade) became a booming business for Europeans and Africans alike, by which African rulers sold their people to Europeans for goods such as iron, alcohol, tobacco and guns. Trans-Atlantic trade led to the degrading use of “chattel” slaves, whereby the slaves were treated purely as property of the owner. The slaves served as sailors, skilled craftsmen or farmers. The journey across the Atlantic, known as the Middle Passage, led to the death of 10-20% of the African slaves. But an even higher percentage lost their lives in the journey from their homes in Africa to the African coast, where they were to board the slave ships.

The Trans-Atlantic slave trade was one component in a system of routes known as the “Triangular Trade” between the Caribbean islands or South America, and New England, and the West Coast of Africa. The three main items that were exchanged were sugar, rum and slaves. European goods, mainly guns, were used to buy slaves from Africa. The slaves were then shipped to the Americas. Then, from America, sugar, rum and tobacco were brought back to Europe, completing the “triangle” of trade. No specific “triangle” of trade has ever been identified and the term was not even used at the time to describe the trade routes; this concept may simply be the result of the imagination of modern historians.

The end of slavery in Europe did not come until the beginning of the 1800s, when Christian reformers such as William Wilberforce and John Wesley began speaking out against the evils of the system. Although the United States banned the slave trade in the early 1800s, slavery in the United States was not abolished until decades later in 1865 when the 13th Amendment was passed, and in South America it was finally ended in Brazil in 1888.

===China’s Role in the Age of Exploration===

In the East, under the Ming dynasty, Emperor Yongle sent a band of explorers led by “Zheng He” throughout parts of southern Asia, Persia, Arabia and Africa. The crew of over 28,000 men embarked on seven naval expeditions in the Indian Ocean from 1405 to 1423 and, as a result, trade with East Africa began. However, by the 1430s, Mongol raids caused the Chinese to withdraw from exploration for the time being, and instead spend their time and money on defenses such as repairing the Great Wall.

The Ming dynasty thereby retreated from the world stage, mostly in order to strengthen Chinese defenses against Mongol threats. Beijing became the capital of China, and eventually the Ming rulers began to retreat even more completely from the outside world, living in an elaborate Beijing palace known as the “Forbidden City.” When a devastating famine occurred in China, the rulers were living extravagantly in their secluded palace and cared little for the plight of the peasants. The people of China revolted, and in 1644, the Ming Dynasty was replaced with the Qing, made up of the Manchu peoples.

Under the Qing dynasty, marriage between members of the Chinese and Manchurians was forbidden, and the official spoken language remained Chinese. Female infanticide occurred, as well as the practice of binding the feet of rich girls and, increasingly, of middle class girls also. The primary philosophy was Neo-Confucianism, a mixture of Buddhism and traditional Confucianism. Jesuits attempted to evangelize China, but were not very successful. At this time all Catholic liturgical services were required to be in Latin, a language that did not fit well with the very different Chinese culture. Nevertheless, a Jesuit presence in China contributed to the exchange of ideas and technology between the East and the West.

China did not withdraw completely from world trade during the Ming and Qing dynasties. China continued to trade with the Spanish, Portuguese, Japanese and Dutch, mainly for silver, which had become the Chinese currency, and with the Russians for furs. Under the Ming dynasty, beautiful blue and white porcelain was produced and exported. The influence of the Ming vase spread throughout the Muslim and Christian worlds, as seen in Holland through the production of Delft pottery; in England, Wedgwood China became the finest producer of china. Other Chinese exports included silk and tea, and imports consisted of spices, textiles and animal skins from Europe. Maize, sweet potatoes and peanuts were brought over from the Americas and became popular crops in China.

==Growth of Nation-States and Monarchs==

The development of European nation-states continued throughout the Renaissance and Age of Exploration. In the 1400s and 1500s, the most powerful nation-states were England, France and Spain. The Holy Roman Empire, which consisted of parts of Italy and Germany, was a powerful but declining force. Enemies that weakened the Holy Roman Empire included the Ottoman Muslims and the French, who resisted an expansion by the Holy Roman Empire. The Reformation contributed to the downfall of the Holy Roman Empire, as did the fact that the cultures of Germany and Italy were too different to remain easily within the same empire.

The governments of the European nation-states grew in power by expanding their central authority. They increased bureaucracy, the size of their armies and the amounts of their taxation. The rise of absolutist rulers contributed to the centralization of France, Spain, Austria, Russia and Prussia. The nature of an absolute monarch was expressed well in this famous quote attributed to Louis XIV (1527-1598), “L’etat c’est moi!” (“I am the state”). Absolute monarchs controlled all aspects of government and society including the church, the bureaucracy and the military. Absolute monarchs claimed “divine right” to rule, stating that they had been appointed by God Himself and were therefore responsible to God alone. They enjoyed “sovereign immunity” under the view that “the king can do no wrong,” a principle that continues to exist in English and American law today (unless government “waives” its immunity with a law granting limited rights to sue it).

Religious conflict in England between Catholics and Anglicans caused absolutism to fail there. The “Glorious Revolution” (so named by supporters of the Church of England) brought down the Catholic King James II and the idea of divine right along with him, placing William and Mary on the throne in 1688. In France, however, absolutism triumphed until the French Revolution, especially with the reign of Louis XIV, the leading absolute monarch in Europe. The French court at Versailles during the reign of Louis XIV, who called himself the “Sun King,” was considered rich in culture and an example to other societies. Truth be told, it was also extravagant and immoral.

In Spain, Philip II (1556-1598), who was the great-grandson of Isabella and Ferdinand, worked to keep Spain Catholic, and viewed Elizabeth I of England as his main rival. But his mighty Spanish Armada was defeated by a quicker English fleet in 1588. In 1579, William of Orange led the Netherlands in a revolt against their Spanish masters. These two losses contributed to the weakening of Spanish power.

Meanwhile, artistic pursuits flourished during this period in Spain. El Greco (1541-1614), a painter, architect and sculptor best known for his paintings of saints and martyrs. as well as a stunning drawn image of a storm over a Spanish town (“A View of Toledo”). Diego Velázquez painted portraits of the royal family and scenes from daily life at court. In 1605 Miguel de Cervantes wrote the novel Don Quixote, probably the most influential piece of Spanish literature, and Lope de Vega wrote an estimated 2,000 plays.

In Russia, absolute emperors known as “czars” arose. The first czar of Russia was Ivan IV “the Terrible,” who executed many innocent people during his fearful reign from 1547 to 1584. Ivan was succeeded by his son Feodor, whose repeated failures led Russia into a “Time of Troubles.” In 1613, the first of a long line of Romanov czars, Mikhail Romanov, took the throne. Under Romanov rule, serfs lost almost all of their freedoms. Peter the Great, another Romanov who ruled Russia from 1682 to 1696, instituted sweeping reforms in Russian government and society. He built St. Petersburg and tried to make European traditions a part of Russia. In the Great Northern War against Sweden, new territories were gained. Other absolutist rulers arose, such as the Hapsburg Empress Maria Theresa in Austria, and the Hohenzollern Frederick the Great from Prussia. These two rulers clashed in the War of the Austrian Succession, during which Frederick won the territory of Silesia.

Europeans during this era desired a “balance of power” that would prevent any single nation from becoming most powerful. The expression of this idea was seen in the intense competition and warfare between the various European nations for control in Europe, especially with the Thirty Years’ War. The war initially began as a religious conflict in Germany between Bohemian Protestants and Hapsburg Catholics in 1618. However, it soon became a political conflict for power that involved Spain, Holland, Denmark, France, Switzerland and The Holy Roman Empire. The Treaty of Westphalia finally marked the end of the Thirty Years’ War in 1648. The war significantly weakened Germany and the Holy Roman Empire, as well as Austria and Spain. Although France was largely a Catholic country, it had chosen the side of the winning Protestants in order to oppose the Holy Roman Empire, and France emerged strengthened and more unified as a result.

==The Scientific Revolution==

A remarkable Polish scientist named [[Nicolaus Copernicus]] (1473-1543) made a stunning claim that rocked the religious and scientific world: he asserted that the earth revolved around the sun, not the sun around the earth. “Finally we shall place the Sun himself at the center of the Universe. All this is suggested by the systematic procession of events and the harmony of the whole Universe, if only we face the facts, as they say, ‘with both eyes open.’” With this bold declaration, Copernicus began the “Scientific Revolution,” which rejected geocentric (earth-centered) theory of the universe dating back over a thousand years to Ptolemy.
[[Image:Copernican model from book.jpg|right]]
Copernicus worked for years on his book describing his theory, and did not publish it until near the very end of his life. He entitled it, “De revolutionibus orbium coelestium” (“On the Revolutions of the Heavenly Spheres”), and published it in 1543. The work was far from perfect: Copernicus assumed that the planets moved in circular revolutions (they actually revolve in ellipses, though the orbit of the earth is nearly circular), and Copernicus also thought that the sun stood motionless as some kind of stationery fixture in the universe (the sun actually moves rapidly too). He also thought the sun was at the center of the universe. A diagram from his book displaying circular planetary orbits around the sun is to the right.

The reception to his work was initially positive within the Catholic Church, which Copernicus served, but to Protestants it conflicted with literal interpretations of the Bible, such as the account of how Joshua benefited from the sun standing still as it passed over the earth. “And the sun stood still, and the moon stayed, until the people had avenged themselves upon their enemies. Is not this written in the book of Jasher? So the sun stood still in the midst of heaven, and hasted not to go down about a whole day.”<ref>Joshua 10:13.</ref> There were few Protestants in Poland then (or now), and thus Copernicus died without much controversy.

[[Johannes Kepler]] (1571-1630), a Protestant in Germany, built on Copernicus’s work and discovered that planets orbited the sun in ellipses rather than circular orbits. Kepler was a brilliant mathematician, astronomer and devout Christian who cited God many times in all of his writings. He felt it was his Christian duty to understand the creation of God, the universe. He also felt that man, being in the image of God, was fully capable of understanding the universe. Like Plato and Pythagoras, Kepler felt that God must have created the universe according to a mathematical plan.

[[Galileo]] (1564-1642) was a contemporary of Kepler who lived in Catholic Italy. Galileo was not as bright as Kepler or Copernicus, and was a bit of an entertaining showman who made a lively party guest. He taught geometry to medical students so that they could use astronomy and astrology in the practice of medicine! Galileo’s primary scientific contribution was his improvement on the telescope in order to view celestial bodies.

But by this time the Catholic Church was in a deadly struggle with Protestantism, which was critical of Copernicus’s theory of the earth revolving around the sun. Wars were being fought and soldiers were dying over religious differences. In 1616, the Catholic Church officially prohibited teaching Copernicus’s theory as fact, although there was no objection to learning it as a mathematical theory. Galileo, however, felt Copernicus was correct and wanted to teach his theory. He had met with Pope Urban VIII on numerous occasions and felt that the Church would not object to Galileo’s publishing the Copernican theory about the universe.

There was no right of free speech in Italy at that time, and in 1632 Galileo published a work called “Dialogue Concerning the Two Chief Systems of the World - Ptolemaic and Copernican” containing Galileo's claim that he had approval of the Catholic Church (an “imprimatur”). Buyers and readers of the book were led to believe that Galileo’s views reflected the official views of the Church, when they did not. Galileo had received permission from a local cleric in Florence, but not from Church authorities in Rome, which would have been required for a work of that significance. Moreover, the Church authorities in Rome felt that Galileo knew that he was prohibited from claiming that the Church supported the Copernican view. This would be like a government official releasing a key military secret with the approval of his boss, but without the higher-level approval that would obviously be required for something extremely significant.

Galileo also angered Church officials in the way that his book mocked opponents of the Copernican system. His “dialogue” was between a fictional Salviati, who supported the Copernican system, and an Aristotelian philosopher given the unflattering name “Simplicio”. The simpled-minded “Simplicio” was made to look like the pope himself, and was mocked in the book for his skepticism about the Copernican system.

In the book’s climax, Salviati supposedly proves the Copernican system by citing the movement of the tides. This theory of Galileo’s was completely false and had already been disproved by Kepler.

Catholic Church officials were furious about this book and the Inquisition banned its sale. Galileo was summoned to Rome. Unlike Luther, Galileo remained a Catholic and fully accepted whatever judgment the Church would render against him. His “trial” was a bit of a farce, probably put on for show more than anything, with Galileo living in plush quarters at all times. At the end of the trial Galileo was required to live in a house where the Church could keep an eye on him during the controversy. Galileo was not imprisoned. Galileo then continued his scientific work but was kept from embarrassing the Church further. Galileo never split from the Church and completely accepted his punishment.

There were other noteworthy scientists during this era. [[Vesalius]] advanced the understanding of anatomy for medical doctors. [[John Harvey]] explained the circulatory system in human anatomy. [[Rene Descartes]] was a French mathematician and philosopher who discovered [[analytical geometry]] and famously declared, “I think, therefore I am.” [[Francis Bacon]] promoted research based on experimentation.

In England there was Sir [[Isaac Newton]] (1643-1727), who after Jesus Christ was perhaps the most influential man who ever lived. If Newton were alive today, then he would be described as a Christian fundamentalist. Everything Newton did was inspired and motivated by his powerful faith in the Word of God in the Bible. Abandoned by his own mother, Newton had an [[inferiority complex]] that drove him to achieve more and more. In one summer he discovered the laws of [[gravity]] and also [[calculus]], and the world has never been the same since.

Like most other men who made great advances in knowledge in world history, Newton discovered and explained something that was unseen: the invisible force of gravity. Newton proposed action-at-a-distance, whereby one object attracts another through an invisible force of gravity. Newton was ridiculed for proposing that something unseen could act at a distance and influence something else millions of miles away from it. Yet Newton was right. Speaking of the world in which we live, Newton once said, “Why not try to understand it?” Newton understood it and explained it to the rest of us, and we are forever grateful.

A German contemporary of Newton, Gottfried [[Leibniz]], deserves mention. Like Newton, Leibniz was a devout Christian. In fact, Leibniz was a Lutheran who dreamed of reuniting that denomination with the Catholic Church. Leibniz was also a brilliant mathematician who developed calculus independent of Newton. It was Leibniz, not Newton, who first published the notation for the integral:
<math>\int_a^b f(x)dx</math> But an ugly dispute developed between Newton and Leibniz over who discovered calculus first. Most English-speaking historians give the credit to Newton but bias may play a role in that. Regardless, the world benefited immensely from Leibniz as well as Newton.

==The Enlightenment==

The “Enlightenment” was a philosophical movement in the 1700s that emphasized an intellectual approach rejecting traditional social, political and (sometimes) religious views. Philosophers in the Enlightenment felt that breakthroughs in science, such as Isaac Newton’s discoveries, could be duplicated in other fields through a systematic and logical approach. However, Newton’s breakthroughs were inspired by his Christian faith, while some of the Enlightenment thinkers rejected and even criticized Christianity. There were Enlightenment philosophers in England, France and even the American colonies. There were stunning new works in classical music, and the arts flourished in what is known as the neo-classical period.

Note how the term “Enlightenment” is biased and self-serving. It makes it appear that people became enlightened compared to their ignorant ancestors. In fact, the achievements of the so-called “Enlightenment” movement were not any greater than other movements and periods in history, and there are valid criticisms of certain aspects of the Enlightenment.

Frenchman Voltaire (1694-1778) was a leading philosopher in the Enlightenment, advocating freedom everywhere and emphasizing his form of reason. He wrote “The Candide,” in which Voltaire described many bad things that happen to Candide in order to make the point that the world could be a better place. Increased freedom was Voltaire’s way of improving things. His ideas later formed the basis of the French Revolution, when many innocent people (including the French royalty and much of the aristocracy) were murdered and churches destroyed amid a reign of terror.

Englishman John Locke (1632-1704) was the leading political philosopher, whose ideas helped the American colonists form a new government. Locke described society as a contract between an individual and society. This was a radical concept at a time when monarchs and the divine right of kings was the controlling theory. Locke’s view helped lay the foundation for the constitutional government that we use in the United States. Locke had built on the prior work of Englishmen Francis Bacon and Thomas Hobbes.

Sir William Blackstone (1723-1780) was the leading legal authority on English law, upon which much American law is based. Law students in America hear about him frequently in law school. In “The Rights of Englishmen,” Blackstone described the source of the rights of the people.

Frenchman Baron de Montesquieu (1689-1755) proposed the concept of separation of powers, and checks and balances, in government. This inspired the United States [[Constitution]]. His book, “The Spirit of the Laws,” explained essential aspects of good government that became enormously influential.

Another Frenchman, Denis Diderot (1713-1784), contributed to literature, speculated on free will and attachment to material things, and edited an encyclopedia of scientific and social knowledge known simply as “Encyclopedia”.

Swiss-born political philosopher Jean Jacques Rousseau (1712-1778) wrote several controversial works. He felt that politics and morality could not be separated, and that the will of the majority was not always correct. Many Americans today would agree with that view. However, Rousseau also attacked private property, and laid the groundwork for future [[communist]] writers such as [[Karl Marx]]. Rousseau declared that government’s goal should be to provide freedom, equality and justice. But note that freedom often results in inequality.

Irishman Edmund Burke (1729-1797) was in the British parliament when the conflict with the American colonies occurred, and he sided with the colonies, making him a hero to many in America. He is also considered to be the world’s first political conservative, and many conservatives today praise him for that reason. His works included “Inquiries into the Sublime and Beautiful” (defending the American colonies and examining how people interpret what they see) and “Reflections on the Revolution in France” (warning, even while everyone else praised the French Revolution, how they were all wrong and how bad the French Revolution was because a few would gain control and abuse their power). What was striking about Burke was that he was almost always right in his political assessments and predictions.

The Scotsman David Hume (1711-1776) was a philosopher and historian who promoted materialism and naturalism rather than spirituality. He was a “skeptic” towards religion, and he penned “A Treatise on Human Nature.” He quipped, “You can tell what is inside a person’s soul by what comes out if it.” Hume is considered the greatest philosopher to write in English, but that is only because there were so few good English philosophers. Hume has been criticized for his atheistic approach and his chief claim to fame is that Charles Darwin declared Hume to have been his central influence, as did “Darwin’s bulldog,” Thomas Henry Huxley. Hume believed in relativism rather than absolute truth, and that not even what God creates has absolute beauty: “Beauty in things exists merely in the mind which contemplates them,” Hume declared.<ref>David Hume’s ''Essays, Moral and Political'' (1742).</ref>

The German Immanuel Kant (1724-1804) was one of the most famous philosophers ever. In his “A Critique of Pure Reason,” Kant criticizes pure reason as a guide to life. Kant may not have been a Christian himself, but he considered Christian values to be the best values in the world. Kant also established a systematic basis for critical philosophy and suggested a material origin for the solar system. Kant’s own suggestion for a moral daily life was this: don’t do something which, if everyone did it, then the outcome would be bad. Expressed another way, an act is moral only if it works as a rule for everyone. For example, littering would be wrong because if everyone did it, then there would be an ugly mess. Kant is taught in all college philosophy departments to this day, but not because he praised Christianity!

Finally, let’s recognize American contributions during the Enlightenment. The greatest works about government were created by the Founding Fathers. These works include the Declaration of Independence (1776) with its statement of inalienable rights and a right to break the social contract with a ruler when he (the king) violates natural rights. The United States Constitution (1787) gave the world a masterful design for government. The Federalist Papers (1788), and particularly articles by Alexander Hamilton and James Madison (most notably No. 10), were also significant.

==Capitalism==

Just as Isaac Newton’s discovery and explanation of the unseen force of gravity was a brilliant insight in science, Adam Smith of Scotland discovered a powerful unseen force in economics known as the “invisible hand.” If government allows free enterprise to flourish on its own, without interference by government, then an “invisible hand” harnesses the power of self-interest for the overall good of society. Companies are guided by this “invisible hand” to work in a way that is beneficial to others, and the overall wealth and progress of society will increase. This is what Adam Smith wrote in the influential ''The Wealth of Nations'', published in the same year as the Declaration of Independence in America: 1776. Adam Smith’s book is still considered the world’s greatest economics masterpiece. He explained how the basis of wealth is found in a free economy with an unregulated exchange of goods, whereby the supply of goods and services responds in an efficient way to the demand of the public without government intervention. Smith advocated the concept of laissez faire, which means “allow to do” or, more simply, “hands off” by government. Ironically, homeschooling in New Jersey (and a few other states) is an activity that most resembles Adam Smith’s ideal of an unregulated market. Let’s defend this lack of regulation of homeschooling while we can!

“[[Capitalism]]” became the prevailing economic theory in Britain as a result of the insights of Adam Smith and others. The Merriam-Webster online dictionary defines capitalism as follows: “an economic system characterized by private or corporate ownership of capital goods, by investments that are determined by private decision, and by prices, production, and the distribution of goods that are determined mainly by competition in a free market.” Britain quickly became the largest and greatest empire in the history of the world as a result of its adherence to this superior economic system. The British empire was larger than even the Mongol empire, except that the British empire was not "contiguous" on land, but included faraway places like Australia and India.

A related economic theory known as “mercantilism” also became popular. Mercantilism was a policy for a nation (such as Britain) to increase its own national wealth based on trade with other nations and territories. Under mercantilism, a nation should accumulate gold by exporting more goods than it imported, and by using colonies (such as the English colonies in America) to ship raw materials to the mother country that could be manufactured and exported to other peoples. Another key component of mercantilism was establishing foreign trading monopolies that would have unfair advantages over competitors in other countries. One such British monopoly was the East Indian Tea Company, and its unfair business advantage caused the colonists in Boston to revolt in the form of the Boston Tea Party.

While Adam Smith’s “invisible hand” was positive in almost every way, “mercantilism” did have a dark side of exploitation of colonies for the benefit of the mother country.

Adam Smith did not invent capitalism, but gave it a powerful intellectual justification. In 1607 -- over 150 years before Adam Smith -- capitalism had already played a prominent role in the first settlement of North America at Jamestown, which was funded by a joint-stock company (a privately owned company having a structure similar to the corporations of today). Joint-stock companies or corporations were (and are) a form of capitalism; decisions are made privately by those acting on behalf of the owners, without any direct control by government.

== References ==

<references/>
[[Category:World History lectures]]
{{DEFAULTSORT:World History Lecture 08}}

Talk:Significance of E. Coli Evolution Experiments

2009-03-06T01:48:42Z

Argon: typos

Talk:Significance of E. Coli Evolution Experiments

2009-03-06T01:46:36Z

Argon:

Talk:Significance of E. Coli Evolution Experiments

2009-03-05T16:06:05Z

Argon:

2008-07-18T01:00:12Z

Argon: /* Formal Response to PNAS */

Now, is this page a report over other people thinking Lenski's paper is flawed or is this Aschafly reporting about himself and putting it in the headline of the Front Page? And please specify by references which "two other" experiments are referenced, and explaing how you can see that the "historical contingency" is not true. --[[User:Stitch75|Stitch75]] 23:22, 12 July 2008 (EDT)
And if you think you argue that well, why dont you submit it as a comment òn the paper to PNAS. --[[User:Stitch75|Stitch75]] 23:24, 12 July 2008 (EDT)

:This article is seriously partisan. I see no outside or third party analysis of the paper in the references, just a bunch of cites to the article or conservapedia itself. [[User:Wisdom89|Wisdom89]] 00:34, 13 July 2008 (EDT)

Folks, you're in the wrong place if you're more interested in who says what rather than determining the truth itself. A true wiki gets at the substantive truth rather than trying to rely on biased gatekeepers and filters of the truth.

The flaws in the statistical analysis in Lenski's paper are clearly set forth and well-referenced. If you're interested in the truth, then look at the paper and see the flaws yourself. If you're not interested in the truth and think you can distract people's attention from it by using other tactics, then you're wasting your time here.--[[User:Aschlafly|Aschlafly]] 00:42, 13 July 2008 (EDT)

I agree with Aschlafly on this, there needs to be some kind of admission or response from Lenski but little has been forthcoming and conservapedia itself has taken it on. Notice how no-one, aside from conservapedia (and I think Creationwiki?) has asked such questions of Lenski? All the magazines etc have taken his study at face value without actually taking the time to critique his claims. Aside from the "peer reviewers" of course.
[[User:JJacob|JJacob]] 00:47, 13 July 2008 (EDT)

: The "peer reviewers" who spent somewhere between 0 and only 14 days looking at the paper, and missed an obvious contradiction between Figure 3 (specifying the "Historical contingency" hypothesis) and Table 1, Third Experiment. The statistical analysis in the paper appears so shoddy to me that I doubt anyone with real statistical knowledge or expertise even reviewed it.--[[User:Aschlafly|Aschlafly]] 00:59, 13 July 2008 (EDT)
::I have expertise in research and statistics and I'm just not seeing this shoddiness that you make reference to. You are allowed to have your doubts, but we should get a bunch of people familiar with such fields to examine the paper's statistical analysis. [[User:Wisdom89|Wisdom89]] 01:01, 13 July 2008 (EDT)
::I have the same problem. i hold a Dr.Rer.Nat title and did some statistics (although i am no expert on it) and fail to see the "shoddiness" please help my underdeveloped mind, Mr. Schafly and enlighten me. I it is so obvious it should be a one-liner to formulate it. --[[User:Stitch75|Stitch75]] 09:54, 13 July 2008 (EDT)

::: Please try harder then. I've expanded the explanations a bit also.--[[User:Aschlafly|Aschlafly]] 10:42, 13 July 2008 (EDT)

:::: At least now i can recongnize your statements more or less clearly. Yet i think (cite from the paper) ''We also used the Z-transformation method (49) to combine the probabilities from our three experiments, and the result is extremely significant (P < 0.0001) whether or not the experiments are weighted by the number of independent Cit+ mutants observed in each one.'' has to be addressed more specifically than you do in order to discredit the statistics used --[[User:Stitch75|Stitch75]] 11:54, 13 July 2008 (EDT)
:::::Just for comparative purposes and a frame of reference, a P value that is less than the significance level of 0.05 is considered significant. [[User:Wisdom89|Wisdom89]] 13:10, 13 July 2008 (EDT)

I wonder, do the PNAS allow questions to raised and asked of the "peer reviewers" themselves? Are we able to find out who/what experience they themselves have? Perhaps that is an avenue that we could look at? I apologise in advance if this has already been asked or answered.
[[User:JJacob|JJacob]] 01:07, 13 July 2008 (EDT)

:Definitly not. Peer reviewers are anonymous and only known to the editor, for good reason. Having peer reviewers non-anonymous would cause reviewers to be very careful to step on nobodys foot to evade revenge. --[[User:Stitch75|Stitch75]] 09:54, 13 July 2008 (EDT)

: No, PNAS probably won't disclose who supposedly did the 14-days-or-less peer review on the [[Lenski]] paper. You're right that such disclosure could shed some light on the final product.

: "Wisdom89", your claim that you "have expertise" and don't see the flaws only makes me conclude that you don't really have the expertise that you claim. Judging by your silly user name, perhaps you've tried that approach before. We're not fooled by it here.--[[User:Aschlafly|Aschlafly]] 01:14, 13 July 2008 (EDT)

(removed silly comment by person who has since been blocked for [[90/10 talk]] violation)

==Marginally significant==
In #1, that's not inconsistent. The figure makes it clear that, according to the hypothesis, the mutations should also occur earlier than 31,000, but should become more common at that point. 12/17 mutants occurred after 31,000. The point in #3 is not clear. What do you mean by ''weighting''? In #5, ''Lenski's paper is not clear in explaining how the results of his largest experiment...his paper refers to his largest experiment as "marginally ... significant," which serves to obscure its statistical insignificance.'' Actually, '''marginally significant''' is clear. It means that the p-value is between .05 and .10 (in this case it's .08, table 2). It's a pretty standard phrase to describe an effect that falls into that range. [[User:Murray|Murray]] 13:41, 13 July 2008 (EDT)

: Your defense of Lenski per point #1 is contradicted by the paper's own abstract, and by the comments by the other Lenski defender below. In point #3, proper weighting is needed to combine multiple studies. In point #5, you don't cite any authority for the unscientific claim of being "marginally significant."--[[User:Aschlafly|Aschlafly]] 20:45, 13 July 2008 (EDT)
: Your defense of Lenski per point #1 is contradicted by the paper's own abstract, and by the comments by the other Lenski defender below. In point #3, proper weighting is needed to combine multiple studies. In point #5, you don't cite any authority for the unscientific claim of being "marginally significant."--[[User:Aschlafly|Aschlafly]] 20:45, 13 July 2008 (EDT)
::Define "proper weighting". Are you referring to how they interpreted the sum of the work or to a specific analysis? You're right, I didn't cite any scientific authority. Here's one: Motulsky, H. (1995). ''Intuitive Biostatistics'', Oxford Univ. Press. Chapter 12. Also, try searching for the phrase in Google Scholar or PubMed, you'll find plenty of uses of it. It's a shorthand way of describing an effect that came close to the arbitrary threshold for statistical significance but did not reach it. [[User:Murray|Murray]] 21:55, 13 July 2008 (EDT)

::: You seem unwilling to accept that Point #1 identifies a clear mistake between the figure and the abstract in the paper, which requires correction by Lenski or PNAS. Given that unwillingness, I doubt it will productive discussing the other mistakes further with you. You're right that other usage of the dubious concept of "marginally significant" can be found on the internet, but the first link to such usage on my internet search for it returned the non-rigorous "Intuitive Biostatistics" and the second link returned a criticism of the concept similar to the criticism express here. [http://annhyg.oxfordjournals.org/cgi/reprint/49/1/93-a.pdf] --[[User:Aschlafly|Aschlafly]] 00:42, 14 July 2008 (EDT)
::::It's not a clear mistake in my mind. There does seem to be some confusion or contradiction in terms of the number of generations, but I haven't seen a clear explanation of why this is or is not a mistake. What seems like a contradiction is the abstract statement that no mutations occurred before 31,500, but it's not clear to me whether I'm misunderstanding that. ''I doubt it will productive discussing the other mistakes further with you. '' Of course you doubt it, because you are unlikely to be willing to concede anything no matter what anyone says. Why do you call the concept dubious? The procedure of determining whether an effect is significant requires the setting of an arbitrary threshold, which is usually .05. That means, in analyses of the sort in the Blount et al. paper, that there's less than a 5% chance that the findings are due to sampling error. When an effect comes close to the threshold it is worth noting, because of the problems inherent in significance testing, which itself is widely criticized in the statistical literature. I am not clear what you mean by weighting, as I mentioned before - in the interpretation or statistically? [[User:Murray|Murray]] 13:30, 14 July 2008 (EDT)

== Richard Lenski incorrectly included generations of the E. coli already known to contain Cit+ variants in his experiments.[3] Once these generations are removed from the analysis, the data disprove Len ==

The author of the above statement must not have read the paper or the supplement carefully. Lenski took clones from those cultures that '''weren't Cit+'''. Careful reading of the paper or the supplement reveals that Cit+ mutants appeared at 750 generations or later into the replay experiment. The authors write: "New Cit+ variants emerged between 750 and 3,700 generations..."--[[User:Argon|Argon]] 15:03, 13 July 2008 (EDT)

: Your explanation is not convincing. Show us how Lenski proved that the samples did not have '''any''' Cit+ variants, if you really think he's claiming that.--[[User:Aschlafly|Aschlafly]] 20:51, 13 July 2008 (EDT)

::I count at least four distinct diagnostic methods used in the paper for distinguishing whether clones are Cit+. One in particular they describe as being very sensitive to weakly citrate-using cells. It's not that I 'think' Blount et al. claim they started with Cit- cells. They say that in the 'Supporting information' document.

::Andy, for tonight, I'll leave it as an exercise for you to answer your own question of how one might test a clone to be sure that is was Cit-. Use the paper if you'd like or present another means. Tomorrow evening I'll cite the methods Blount used in the paper.

::Meanwhile, here's another question to ponder: If the colonies used to start the 'replay' cultures were Cit+ at the start, why is it that no Cit+ cells were found in such cultures before 750 generations? These cultures were all started from single clones which must have either had the Cit+ phenotype or not.--[[User:Argon|Argon]] 18:38, 14 July 2008 (EDT)

-
How did Lenski obtain Cit- clones from later populations after Cit+ had become the dominant phenotype? It's pretty standard procedure. First, keep in mind that these populations contained a minority of Cit- cells. Now, a small liquid growth is made from the frozen glycerol stock of cells from a given generation, say 32,000. This is grown to an appropriate OD and then plated on a standard rich media agar plate. Provided that the plated sample isn't too dense, this plating will deposit somewhere between 15-100 individual cells on the agar, with plenty of spacing between them. The plate is then incubated at 37 C for a given time, during which the individual cells replicate to form small, visible colonies. These colonies will consist solely of cells that are genetically identical to the original cell deposited on the agar. This is all well and good, but how do you find which of the colonies are Cit-? Here you use a technique called replica plating. The agar plate is gently inverted onto a sterile swatch of velvet so that some of the cells from each colony are deposited onto the fabric. Next, a citrate-only agar plate is pressed against the velvet, transferring cells from the fabric to the new plate in exactly the same spacial orientation as the first plate. The cells are then allowed to grow on the citrate-only plate. When we then compare the two plates, we look for colonies on the rich plate that did not grown on the citrate-only plate. These colonies will be Cit-, and can be used for the replay experiments. [[User:Gerlach|Gerlach]] 13:45, 14 July 2008 (EDT)

:Just a correction: The Cit+ cells didn't dominate the cultures at generations 32,000 & 32,500. Cit+ mutants represented 12% and 19% of the population at these times points, respectively. Replica plating is definitely one means of identifying Cit- and Cit+ clones but because the Cit+ cells represented a fraction of the total, it is simple enough to streak the samples to individual colonies (founded by single cells) and test each colony individually. Otherwise, a nice description of replica plating (a technique developed by Joshua Lederberg that allowed him do the research for which he was awarded one of the 1958 Nobel Prizes in Medicine)--[[User:Argon|Argon]] 18:38, 14 July 2008 (EDT)

::Oops, you're right, Argon. Cit+ wasn't dominant at generation 32,000, so replica plating might not be the best option. Still, in the case of almost 20% Cit+, I think replica plating might be labor reducing. I'm a biochemist by training, not a microbiologist, but I'm sure there are other methods that could be used. For example, Christensen's agar appears to provide a sensitive, colorimetric method of identifying even weakly citrate-utilizing colonies, so one might be able to plate cells on a Christensen's agar plate and pick uncolored colonies. Again though, I'm no microbiologist so I don't know the best method. However, there are ways to easily pick Cit- clones from later generations in this case. That said, I think that this specific "flaw" cited in Lenski's paper should be removed from the main article. Clearly, Cit+ were not used in the replays from later generations, so Conservapedia's objection is totally baseless.[[User:Gerlach|Gerlach]] 19:19, 14 July 2008 (EDT)
:::All appropriate descriptions. For generating estimates of the population distribution (Cit-/Cit+) in a culture, replica plating would certainly be applicable. The main point is that identifying and isolating Cit- clones from the generations used in the experiment is straightforward. I agree that it would be a mistake to keep that objection in the article (actually that thread runs across several points in the article).--[[User:Argon|Argon]] 20:12, 14 July 2008 (EDT)

:::: Nothing in the paper rules out contamination of those samples by Cit+ variants, and you quote nothing in the paper to rule it out. There was no reason to use and rely on these samples that already have Cit+ variants.--[[User:Aschlafly|Aschlafly]] 23:04, 14 July 2008 (EDT)
:::::The whole point of the discussion above was to illustrate that there are methods to isolate cells with a particular genotype from a mixed population. Suppose we used the replica plating method (maybe not the best choice, but it's standard procedure). Care to explain where the Cit+ contaminants could possibly come from? And the naivete inherent in objecting that "nothing in the paper" rules it out is simply astounding. Standard laboratory procedures, like isolating specific clones, typically don't show up in papers. For example, if I to write a paper on my research, it would not include a detailed discussion of the construction of my expression plasmid. I might mention that I cloned my gene into a particular vector, but there wouldn't a discussion of how I did my PCR, restriction digests, or transformations. Things like that are simply extraneous details that are taken for granted by experienced researchers. There is simply no reason for Lenski et al. to include a discussion of their method for isolating specific clones. This is why this whole exercise of Conservapedia criticizing Lenski's paper is folly, because most Conservapedia users simply don't know standard laboratory techniques. It's kind of important to know what scientists today are able to do on a routine basis before wading in to claim that they couldn't have done what they claim.[[User:Gerlach|Gerlach]] 09:05, 15 July 2008 (EDT)

:::::***** *****
:::::Actually the methods cited in the paper would separate the Cit+ lines from the Cit-. They took single colonies (founded by single cells), and tested them on selective agar (minimal citrate media - MC agar) and an indicator medium, Christensen's citrate agar (Product information from Sigma here: http://www.sigmaaldrich.com/sigma/datasheet/c7595dat.pdf). As Gerlach notes from the Blount paper, Christensen's citrate agar is very sensitive to citrate utilization. Typical ''E. coli'' strains do not produce a color change but other citrate-using enterics like ''S. typhimurium'' and Cit+ ''E. coli'' mutants appear pink/red on the plates.

:::::This conclusion is further validated (as I mention above), by the fact that Cit+ mutants did not appear immediately in the replay experiments (Others have also noted this). For example, in replay set-1, it took 750 generations before the first Cit+ mutants were isolated. Many didn't produce Cit+ cells after 3700 generations. If the starting line was Cit+, *all* the cells in the culture would have show up as Cit+ in the first pass.

:::::Overall, it is abundantly clear that the cell lines used in the replay experiments were not Cit+ at the beginning.--[[User:Argon|Argon]] 19:44, 15 July 2008 (EDT)

== 1. Lenski's "historical contingency" hypothesis, as specifically depicted in Figure 3, is contradicted by the data presented... ==

Again, more issues with comprehension: The authors did not know when the potentiating mutation first arose but they knew it was before generation 31,500. Figure 2 is merely figurative, being an illustration, and not quantitative. Their analyses suggested that the potentiating mutation did arise at about the 20,000 generation point or later. Their conclusion is that Cit+ mutation rate is low even in a potentiated background but apparently distinguishable from a low-incidence single, unpotentiated event.--[[User:Argon|Argon]] 15:04, 13 July 2008 (EDT)

== Statistics101 package: ==

What does the author imply by writing: "...Lenski himself does not have any obvious expertise in statistics. In fact, Richard Lenski admits in his paper that he based his statistical conclusions on use of a website called "statistics101"?

From the web site:
''Professionals: Although it was originally developed to aid students, the Statistics101 program is suitable for all levels of statistical sophistication. It is especially useful for Monte Carlo, resampling, and bootstrap applications. It has been used by professionals in many fields. These include anthropology, biology, ecology, evolutionary biology, epidemiology, marine biology, psychology, toxicology, veterinary pathology.''
It would appear that the package from the web site (not the web site itself) was used to perform the Monte Carlo resampling tests. Is there any evidence that the package produces incorrect results?--[[User:Argon|Argon]] 19:41, 13 July 2008 (EDT)

==Formal Response to PNAS==

''I've moved this thread from the open section above since this page is being categorized.'' --[[User:DinsdaleP|DinsdaleP]] 21:52, 13 July 2008 (EDT)

:::I think this article presents very sound arguments. Conservapedia should now take action, offering to publish a rebuttal of Lenski in the PNAS journal.--[[User:JBoley|JBoley]] 11:31, 13 July 2008 (EDT)

I agree with JBoley in that if Conservapedia wants to present a formal, professional response to Professor Lenski's paper that questions specifics within his paper, then it should happen. That is the proper execution of the scientific method, and I'm certain that a professional response to PNAS would yield better results than vague "give us all the data " demands. Is a formal response to PNAS from Conservapedia in the works, or is this article the only place these questions/objections were intended to be raised? --[[User:DinsdaleP|DinsdaleP]] 11:52, 13 July 2008 (EDT)

: I don't know if PNAS would embarrass itself by printing a rebuttal, or whether it has the integrity to retract Lenski's paper. Conservapedia's audience is probably bigger than PNAS's, and we're certainly not going to suspend our exposure of the truth here in order to await correction by PNAS.

: PNAS publishes, quite easy to find Rules for "Rebuttals" (they call it "Letter"). It is specifically what you want. [http://www.pnas.org/site/misc/iforc.shtml PNAS:Information for Authors] . I cite: ''Letters are brief online-only comments that contribute to the discussion of a PNAS research article published within the last 3 months. Letters may not include requests to cite the letter writer's work, accusations of misconduct, or personal comments to an author. Letters are limited to 250 words and no more than five references.'' --[[User:Stitch75|Stitch75]] 13:25, 14 July 2008 (EDT)

: In addition, Lenski has already demonstrated how he reads this site and he can certainly correct his own paper, and he should do so. Indeed, professionalism might support giving Lenski the time to correct it himself first.--[[User:Aschlafly|Aschlafly]] 12:14, 13 July 2008 (EDT)

::I understand what you're saying, but there's nothing improper or unprofessional in submitting a formal request to PNAS to have the points in this article addressed by Professor Lenski and his team. To be frank, you've been adamant in your insistence that PNAS has been less than rigorous in the review of Lenski's paper, so if one of your intentions is to demonstrate this then having PNAS respond to a formally submitted response to the paper in public would serve that purpose. This can be done in addition to publishing these objections on Conservapedia--[[User:DinsdaleP|DinsdaleP]] 12:21, 13 July 2008 (EDT)

:::If not PNAS, then perhaps some other public forum? I know Andy Schlafly has appeared on television, effectively arguing against gardisal and other dangerous vaccines. Perhaps if a TV program were interested you could argue against Lenski? You could be the spokesperson against these false claims of evolution.--[[User:JBoley|JBoley]] 12:24, 13 July 2008 (EDT)

:::: I'm not opposed to the above suggestions, but the future is here, folks. Lenski, PNAS editors and television producers have free will to reject or ignore the truth, and I'm more interested in getting the truth out here than trying to persuade someone in dying media like print or television. Lenski and his defenders can see the truth here, and they can decide for themselves whether to reject or admit it.--[[User:Aschlafly|Aschlafly]] 12:30, 13 July 2008 (EDT)

:::::I agree with you about the dying nature of print (I don't think television is dying, merely changing). The problem is that information about the flaws in Lenski's study are not registering outside of sites like Conservapedia. In effect, Conservapedia is an echo chamber. People that come to this site already agree with its point of view. I encourage you to attempt to attract the attention of other forms of media, or Lenski's false claims will simply be accepted as fact by the public and even worse, by educators.--[[User:JBoley|JBoley]] 12:36, 13 July 2008 (EDT)

::::::While there's nothing wrong with taking one's message to various forums or outlets, I believe there's a specific value in submitting these objections as a formal response to PNAS. Conservapedia was established as a trustworthy resource for students, and in my mind all of it's actions should be done with the goal of informing and educating. The Lenski debate is over the findings published in a scientific journal after undergoing a peer-review process. The objections to this paper by the CP leadership are not just about its content, but to the process by which it was reviewed and published in the timeframe it was. Talking about these objections is fine, but it's more instructional to the students using Conservapedia, and a better example of the scientific method in action, to respond to a scientific paper published in a journal through the formal process by which such papers are either defended or corrected. In the end, Lenksi's work will either stand up as good science, or any errors will be addressed and the paper's conclusions modified accordingly, which is also good science. Seeing this process in action regarding a such a significant paper is a great learning opportunity, and the Conservapedia leadership would be remiss in not standing by their conviction in these objections and submitting them formally to PNAS. --[[User:DinsdaleP|DinsdaleP]] 12:38, 13 July 2008 (EDT)

PNAS has a letters section available with the online edition. Many journals have a letters section for rebuttals or clarification. Legitimate corrections are welcome. Andy, have you run your list of 'flaws' past any biologists?--[[User:Argon|Argon]] 15:36, 13 July 2008 (EDT)

: Funny, Argon, how you don't apply your demand of expertise to Lenski himself. What are Lenski's credentials with respect to statistical analysis? Has he even taken and passed an upper-class statistics course of any substance?--[[User:Aschlafly|Aschlafly]] 00:45, 14 July 2008 (EDT)

::I don't follow. Are you suggesting that Lenski doesn't understand the proper application of the statistical methods used in his paper? If so, I haven't seen a description of which alternate methods you'd employ, let alone any output from such an analysis. Here's a thought: Why don't you substantiate your claims by writing up the work and submitting it as a correction letter to PNAS?--[[User:Argon|Argon]] 18:46, 14 July 2008 (EDT)

::: Argon, if you really "don't follow," then try harder. You insist on credentials by others who criticize Lenski, and yet you do not insist on expertise by Lenski in statistics with respect to his "analysis". Perhaps Lenski should first take and try to pass "Statistics 101" before trying to use a website by its name to draw flawed conclusions.--[[User:Aschlafly|Aschlafly]] 22:36, 14 July 2008 (EDT)

::::Do you really think that Lenski went to statistics101.net to learn about statistics and how to apply them? That simply is not the case. The Lenski group knew that they needed to do a Monte Carlo resampling analysis on the results of their replay experiments. In this situation, they are faced with two choices: either code an appropriate program themselves, or utilize one that has already been developed and is readily available to researchers. Since statistics101 had such a program available, they chose the latter option. Statistics101.net was simply the source of the program that the Lenski group used to perform the statistical analysis. If you want to argue against that choice, then you need to examine the source code for the statistics101 package and enumerate why it should not have been used.[[User:Gerlach|Gerlach]] 11:40, 15 July 2008 (EDT)

::::***** *****
::::I don't insist on credentials per se. I *recommend* developing a working understanding of the experiments or a willingness to do the necessary background research to get the details straight before heading off on possibly the wrong direction. It saves a lot of thrashing about. It's perfectly OK to raise questions but before leveling accusations it might be nice to do that privately and discuss that with others who can provide useful feedback. Just my 2 cents.--[[User:Argon|Argon]] 19:48, 15 July 2008 (EDT)

::Mr. Schlafly, does your response above, "I'm more interested in getting the truth out here than trying to persuade someone in dying media like print or television", mean that you will not be submitting a response with these identified flaws directly to PNAS? As I mentioned above, it takes nothing away from the value of posting these statements here on CP to also submit them to PNAS, but the proper way to prompt a journal to review and correct an article is through a direct response, not publication on an unrelated website like CP. It's not proper for anyone but the author(s) of the objections in this article to make that submission, so I'm hoping they step up with the conviction of their beliefs and respond to PNAS directly. Thanks. --[[User:DinsdaleP|DinsdaleP]] 12:32, 14 July 2008 (EDT)

::: By now Lenski has probably seen the flaws identified on the content page here. What's "proper" is for him to correct his own paper in PNAS. The criticism will likely continue as long as he declines to do so. If anyone here would like to educate the PNAS editors about the flaws, then please feel free to do so.--[[User:Aschlafly|Aschlafly]] 22:36, 14 July 2008 (EDT)

::::I'll be glad to submit these items to PNAS on behalf of Conservapedia using the "Letters" forum described by Stitch75 above. Should I cite you the author of this analysis, and is there an email you'd prefer me to include instead of my personal one for any PNAS response? I'll put up a draft of the Letters submission here for your approval before sending anything out. Thanks. --[[User:DinsdaleP|DinsdaleP]] 18:39, 15 July 2008 (EDT)

:::::It would have a very strange scientific taste should somebody else that Mr. Schlafly be the first author of the letter (however, if you insist on that, i suggest "personal communications" as the right kind of citing the work here.). It is his turn to stand up to his claims in this way (by publishing it into a Forum which he is not the owner of). I personally share the doubts of many here about his argumentation (and his understanding os the experiments), but i am sure only more of his dismissive comments will follow. However i see that he comes up with a clear alternate Hypothesis (contamination), so he is free to show that this is more likely (calculations please) and peform Monte-Carlo simulations on it (for a person complaining that statistics101 is to simple that should be no difficult task). As far as i understood and see the data Lenski and coworkers did the best to exclude this Hypothesis, however i did not run own simulations (And I won't do it, because i think nothing will come out - furthermore Mr. Schlaflys personal style in the communication "you have to try harder" is not the style i am used to be adressed by people whose qualification in a subject is appearlingly nor more than mine). So running the simulation, evaluating his own hypothesis using a valid statistical method is now Mr. Schlaflys job - if he come up with a decent calculation showing this Hypothesis is more likely, the letter would for sure be accepted and Lenksi would have to react. If Mr. Schlafly is not the first author of the letter he could evade the critics after that by saying that he was misunderstood, which means somebody else take the risk of submitting the letter, but in case of success Mr. Schlafly would take the glory. --[[User:Stitch75|Stitch75]] 20:10, 15 July 2008 (EDT)

::::::I would prefer that Mr. Schlafly submit his objections to PNAS directly as well, but since he has declined to do so the next best response is to submit it "on behalf of Conservapedia", which he has authorized above. I'll post the draft letter tomorrow, and it will credit him as the author unless I'm asked to include other individuals who contributed to the analysis. --[[User:DinsdaleP|DinsdaleP]] 22:05, 15 July 2008 (EDT)

:::::::I hope ASchlafly does offer a rebuttal of Lenski's flawed "study." However, if he does not, I am all for DinsdaleP's suggestion. I look forward to reading your draft. If you take all the objections that Conservapedians have raised to Lenski's paper, I do not see how PNAS can possibly object.--[[User:JBoley|JBoley]] 11:28, 16 July 2008 (EDT)

::::::::They could object because Conservapedia's criticisms are obviously false. If this page hasn't made that clear, I don't know what could.[[User:Gerlach|Gerlach]] 13:44, 16 July 2008 (EDT)

:::::::::They would have to prove that Conservapedia's criticisms are false, and they cannot do that. You sound like a Lenski supporter. You need to open your mind to the truth.--[[User:JBoley|JBoley]] 13:59, 16 July 2008 (EDT)

:::::::::Gerlach, the most obvious lesson from this page is how a few, yourself included, seem determined to defend a flawed paper no matter what the truth brings. You have free will to reject whatever you want, but you're only hurting yourself by that approach. People who do open their minds are amazed by the insights and happiness it brings.--[[User:Aschlafly|Aschlafly]] 21:54, 16 July 2008 (EDT)

:::::::::This is such a strange response, Andy. The attitude that you and several of your defenders seem to have is that the criticisms you have made of Lenski's paper cannot be rebutted. However, the fact that you ''can'' level criticism at something doesn't make that criticism true. That is, you may be wrong. And that is the case here. Myself and others have pointed out that your alleged flaws in Lenski's paper are incorrect, and based on misunderstanding it, or, even worse, simply not reading it carefully. There has been little to no substantive response to our detailed rebuttals. Replies from you or other defenders of the Conservapedia article amount to little more than brazen declarations that we are wrong or "nonresponsive", and that criticisms in the original article remain unscathed. But for people who claim to have "the truth" on their side, this is simply baffling. If the veracity of your claims against Lenski's paper is so obvious, then it should be an easy task to provide in depth responses to the points that we have been raising against your article. I'm not above criticizing scientific papers. Bad papers get published frequently, after all. However, Lenski's paper does not appear to be one of those. Notice, though, that I am not claiming it to be perfect, no paper is. That said, whatever flaws the paper may have, those presented in the Conservapedia article are not among them. [[User:Gerlach|Gerlach]] 11:43, 17 July 2008 (EDT)

:::::::::: Gerlach, now you seem to admit that the Lenski paper may be flawed, but that Conservapedia has not identified any of those flaws! With all due respect, you seem to have taken closemindedness to new heights.

:::::::::: Lenski claims there is a mutation rate, yet his presented data show that the number of mutations do not scale with sample size. His presented data disprove both of his hypotheses.--[[User:Aschlafly|Aschlafly]] 13:27, 17 July 2008 (EDT)

:::::::::: First, we need to clarify what we mean by "flaw". Are we talking about serious methodological flaws or errors in interpretation that fatally undermine the paper's conclusion? If so, I fail to see any such flaws. Of course, any honest scientist recognizes that any work at the frontier of science has the potential to be wrong. It ''may'' be that future work shows Lenski to be wrong. But let's not confuse this possibility with actuality: saying that something might be wrong is not the same as saying that it is. On the other hand, by "flaw" we may mean minor experimental details that could have been better. In this case, the paper is flawed, and Lenski himself admits this in the supporting information. But ''every'' paper is flawed in this manner, and I doubt you'll find any investigator who wouldn't say that they wish they had done some things differently during the course of their research. As an example of what I'm talking about, the Lenski group's statistical analysis would have been improved if they could have accounted for the evolution of increased cell size (and, therefore, decreased cell density) in later generations. As it stands, their analysis ''underestimates'' the potentiation effect in these generations because replays of later generations involved fewer cells. But this flaw, and other flaws of this type, do not undermine the conclusion of the paper.

:::::::::: Regarding the mutation rate argument, see the earlier discussion on this page. The third experiment was significantly different than the first two. The experimental scheme was different, it was performed at at different time and with different conditions, and utilized, for the most part, different clones than the first two. And Lenski doesn't just claim there is a mutation rate in potentiated cells, he ''actually measures it''. But frankly, I don't understand your argumentation here. It seems that you are trying to suggest that all the Cit+ mutants isolated were the result of contamination. This clearly isn't the case, for several reasons. However, this isn't the place for this particular discussion, as there is already such a discussion elsewhere on this page. I still haven't seen any adequate response to our points against the Conservapedia article, and I don't think I am closed-minded for expecting such a response. As I said, I'm open to the possibility that the Lenski paper is flawed, but I expect cogent argumentation to support any such flaws. I haven't seen that here.[[User:Gerlach|Gerlach]] 15:09, 17 July 2008 (EDT)

::::::::::: Gerlach, look at how many words you used to sidestep my simple explanation of a flaw in Lenski's work: "Lenski claims there is a mutation rate, yet his presented data show that the number of mutations do not scale with sample size. His presented data disprove both of his hypotheses." If the third experiment of Lenski's was independently flawed as implied by your response, then that does not help your defense. Note that in Lenski's second experiment the mutations also failed to scale with sample size.--[[User:Aschlafly|Aschlafly]] 18:26, 17 July 2008 (EDT)

::::::::::::I think there is a misunderstanding. Blount et al. did not say that the absolute mutation rate was constant for the generation of Cit+. After all, they found that variations in culture conditions had some effect. The replay experiment was performed to determine if clones from later cultures were more or less likely to give rise to Cit+ cells than clones from earlier generations. The authors' hypothesis was that a potentiating mutation was required before the Cit+ mutations could arise. If that was the case then clones from later generations would be more likely to produce Cit+ cells. If the Cit+ capability was the result of an extremely rare, single mutation, then any generation of clones would be equally likely to produce Cit+ cell. The hypothesis is that in a particular experiment, the '''relative probability''' of generating a Cit+ mutant would be greater with clones from later generations. '''Absolute mutation rates''' (which appear to be Andy's concern) may be contingent on the growth conditions, which differed between the three replay experiments but '''''within any particular set of conditions''''', one might expect the relationships between Cit+ recovery and the generations from which the starting clones were derived would still hold.

::::::::::::pg 7902 of the paper in the journal:
::::::::::::"According to the rare-mutation hypothesis, Cit+ variants should evolve at the same low rate regardless of the generation of origin of the clone with which a replay started. By contrast, the historical-contingency hypothesis predicts that the mutation rate to Cit+ should increase after some potentiating genetic background has evolved. Thus, Cit+ variants should re-evolve more often in the replays using clones sampled from later generations of the Ara-3 population."

::::::::::::From the abstract:
::::::::::::"The long-delayed and unique evolution of this function might indicate the involvement of some extremely rare mutation. Alternately, it may involve an ordinary mutation, but one whose physical occurrence or phenotypic expression is contingent on prior mutations in that population. We tested these hypotheses in experiments that ‘‘replayed’’ evolution from different points in that population’s history. We observed no Cit+ mutants among 8.4 x 10^12 ancestral cells, nor among 9 x 10^12 cells from 60 clones sampled in the first 15,000 generations. However, we observed a significantly greater tendency for later clones to evolve Cit+, indicating that some potentiating mutation arose by 20,000 generations. This potentiating change increased the mutation rate to Cit+ but did not cause generalized hypermutability."

::::::::::::What the authors found was that the later generations really did produce more Cit+ mutants than you'd expect if the Cit+ mutation was instead randomly distributed (or, as Andy has claimed and not yet retracted, contaminated by Cit+ cells). Yes, the absolute rates didn't scale across the three different conditions but within each experiment it is clear that the Cit+ mutants arose from cells taken at later generations. Yes, there were differences in the absolute rates under different conditions but that doesn't mean the results and conclusions about potentiated clones are wrong.

::::::::::::As for calculations of mutation rates: Blount et al. performed additional fluctuation experiments (journal pg. 7903) in an attempt to estimate the relative effect of the potentiating mutation and to calculate a rough estimate of the mutation rates. Keep in mind, those rates '''''are referenced to growth under the specific conditions used in that particular experiment''''' and were used to provide ballpark estimates for comparison to other classes of known mutations. As for the second experiment not 'scaling' (with the first?), I wouldn't expect that. The first involved clones grown in continuous liquid subcultures and both the total number of generations, cells/generation and growth conditions (e.g. liquid with nutrient replenishment vs. solid agar) are very different.--[[User:Argon|Argon]] 19:20, 17 July 2008 (EDT)

::::::::::: I have not "sidestepped" your point, Andy. Your claim is that the number of Cit+ mutants obtained did not scale with sample size in the third experiment. In response, I reiterated what Argon and I have said previously, specifically that there are differences between the third and second experiments that make a straight comparison between the two inappropriate. That said, if anyone is sidestepping the issue, it is you. Argon and I have addressed your objection previously, but your only response has been to stubbornly restate the original claim. If you don't agree with our statements, then you must explicate why. Additionally, I did not say or imply in any way that the third experiment was flawed. I said that it was significantly different from the second, but this is not the same as saying it was flawed. Argon has provided a response to the scaling of the second experiment to the first.[[User:Gerlach|Gerlach]] 20:21, 17 July 2008 (EDT)

:::::::: In response to earlier postings about drafting and submitting a letter to PNAS, I'm for it and would be happy to contribute. In response to a comment above, I don't want any "glory" and learned a long time ago that nobody gets credit or money for telling the truth. More often those who speak the truth are reviled and insulted, but mockery doesn't bother the truth as much as it bothers falsehoods.--[[User:Aschlafly|Aschlafly]] 22:01, 16 July 2008 (EDT)

:::::::::Although I disagree with most of Andy's conclusions, I am in favor of Andy signing and submitting his comments to the PNAS on behalf of Conservapedia (and the Eagle Forum , assuming they provide some sponsorship of this site). That is, after all, within his rights and possibly a responsibility as well. Given that Andy outlined most of the pages on the subject and authored most of the core arguments on the pages, I think it would be crazy for Andy to walk away from what has clearly been his leadership role.--[[User:Argon|Argon]] 19:27, 17 July 2008 (EDT)

== Expertise in Statistics ==
I don’t mind admitting that I have none! The article states that Lenski combined the three trials incorrectly and that he doesn’t make the insignificance of the third clear. Now it may well be very clear to others but, unfortunately, the article doesn’t make clear to people like me what was wrong. Can someone expand on this point? Is anyone able to explain to a layman what Lenski should have done and the conclusions he should have reached? It's all getting a little technical for me. --[[User:Billd|Bill Dean]] 12:10, 14 July 2008 (EDT)

: OK, here's point 4 in layman's terms: Lenski's hypotheses of a mutation rate imply that a ten-fold increase in sample size should result in a ten-fold increase in mutations. But it doesn't. In fact, a nearly ten-fold increase in sample size results in only a slight increase in mutations in Lenski's data. These data, as presented by Lenski in his paper, suggest (if properly interpreted) that there is no mutation rate at all. Rather, these data are more consistent with occasional contamination, broadly defined.--[[User:Aschlafly|Aschlafly]] 23:15, 14 July 2008 (EDT)

::Except that the growth pattern of the Cit+ cells in the third replay experiment demonstrated that there were no Cit+ cells in the cultures at the time of plating. Control plates with a mix of Cit- and Cit+ cells were done which showed rapid development of Cit+ colonies, whereas the experimental plates did not show any Cit+ colonies for at least 8 days and up to 28 days. If you believe that the plates became randomly contaminated during the course of the incubation, rather than at the initial plating, the post-plating contamination should have affected all generations equally, but it did not--[[User:Brossa|Brossa]] 12:46, 15 July 2008 (EDT)

::Blount et al. discussed the unexpectedly low Cit+ conversion rate for the third replay set in the supplementary document. They have some speculations but don't know why it occurred. They are dealing with low probability events. I don't agree that the 'proper' interpretation is contamination. First, they isolated and tested each Cit+ isolate for the markers and sequences we've discussed on earlier Conservapedia pages. Second, the distribution of Cit+ isolates does not appear to be random: They correlate strongly with the later generations. If undetected contamination was random, it's unlikely the Monte Carlo resampling tests would reject the null hypothesis of the 'rare-mutation' hypothesis over the 'potentiated' hypothesis. As Brossa correctly notes, contamination would have affected all generations. Blount et al. write in the paper's supplement: "To facilitate handling and minimize possible confounding variables, we divided this third experiment into 20 blocks of 14 clones each. All of the clones within a block came from different generations, and the single ancestral clone was included in all 20 blocks."--[[User:Argon|Argon]] 20:01, 15 July 2008 (EDT)
::: Good and detailed analyis and conclusion, Argon. I had no time to read everything in that detail, but i think you present exactly the arguments (contant background would favour rare mutation hypothesis) would be exactly the one i had in mind, so let's see how Mr. Schlafly will bend his own Monte-Carlo simulations to showing several 1000 times no mutation into accordance with any reasonable background explaining a significant number of counts in *some* samples --[[User:Stitch75|Stitch75]] 20:20, 15 July 2008 (EDT)

:::: The above comments are non-responsive. The essence of ''both'' of Lenski's hypotheses is that there is a fixed, or stepped, mutation rate. But any such rate would be roughly proportional with sample size. Yet his three experiments prove otherwise, which Lenski fails to address in a satisfactory manner. Indeed, Lenski's presentation of his data disproves the very thing he claims to have shown.--[[User:Aschlafly|Aschlafly]] 20:40, 15 July 2008 (EDT)

::::: Mr. Schlafly, please answer: Which the of the two hypotheses is, according to your calculations the most likely? --[[User:Stitch75|Stitch75]] 20:47, 15 July 2008 (EDT)

::::::The first replay experiment was run under different conditions - roughly 3700 generations in continuous subcultures. One can't compare population sizes or rates between the first and last two replay experiments. As I noted, Blount et al. acknowledged the possible anomaly of lower than expected mutants in the third run. Still, the second and third experiments were run under different conditions. The plates in the second replay experiment were seeded with fewer cells per plate than in the third experiment. That change can affect the survival rates of cells on plates over time (e.g. different rates of nutrient exhaustion). Given the extremely low mutation rates involved, there is no simple means of normalizing the numbers of mutants recovered for the second and third experiments. One might expect 'roughly' 10x more mutants in the third experiment but that's truly a 'rough' estimate that would be affected by conditions under which cells are exposed. It's certainly a question that remains and they may be able to clear up with future research. In any case, the pattern of data does not support Andy's claims of contamination or that the hypotheses of the paper is in error.--[[User:Argon|Argon]] 21:27, 15 July 2008 (EDT)

::::::: Argon, you have free will and it's clear to me that you are going to exercise it to the point of embracing absurdities. This time you claim that "extremely low mutation rates" would not result in mutations that scale with sample size (of course they would), and that density completely alters mutation rate (if that silly claim were true, then Lenski's experiment was flawed from the get-go). Your belief system is remarkable, but it's not logical.--[[User:Aschlafly|Aschlafly]] 23:38, 15 July 2008 (EDT)

::::::::Fair enough, but let's be clear: That's my take on the paper and that of most of the scientists who reviewed the paper, read the paper and commented upon the paper so far. For that matter, Michael Behe didn't call the data flawed, nor did those commentators at Dembski's Uncommon Dissent blog, nor did Dr. Georgia Purdom at Answers in Genesis. In contast, those who think the work is flawed appears to be limited mostly to you, Andy. It is true that 'scaling' was not seen in this case but as we've seen, the conditions were not quite the same and it is known that this can have an impact.

::::::::However, the fact that the conditions were not identical doesn't detract from the fact that the emergence of Cit+ clones *still* correlated with the sampling of the later generations. What this means is that with three separate experiments and under three sets of conditions, the constant-rate-mutation hypothesis doesn't hold. What the differences between the second and third replay experiments demonstrates is that they were run under different conditions that affected the overall rate of conversion, not that the 'potentiated mutation' hypothesis is wrong. Those are actually distinct questions. Andy, the data simply does not support your claims that the cultures were contaminated (we'd expect random distribution), or that the 'scaling' variations ruined the experiment. In my opinion, you seem focused on red herrings to the exclusion of evaluating the data in the overall context of the experiment which demonstrates a correlation of Cit+ clones emerging from samples taken at later generations. Would you care to address that pattern and discuss why your 'contamination hypothesis' doesn't appear to hold up?--[[User:Argon|Argon]] 09:29, 16 July 2008 (EDT)

:::::::There is nothing absurd in what Argon said. As he mentioned, the third experiment was performed under different conditions and at a different time than the first or second experiments. The key here is that there are plenty of variables, some unknown, that simply aren't controlled for between the experiments. For example, take the fact that it was performed at a different point in time than the first two. The third experiment, then, is almost certainly being performed with different batches of growth media, liquid and solid. Anyone who has spent any significant time growing cells knows that media can vary significantly in growth characteristics between batches, despite the same recipe being used. The reasons for this can be many. Perhaps the balance or pH meter was off calibration one day, or a different bottle of reagent was used. Take an analytical chemistry course, you'll spend plenty of time talking about this. It is true that, for the most part, this sort of variability has minimal impact. But when you're examining something like an extremely rare mutation, or you're trying to make an extremely accurate measurement, intra-lab variability like this can be significant. For an extremely rare mutation like Cit+, which involved at least two additional mutations in potentiated cells, any change in mutation rate can have a significant effect on your ability to obtain mutants. Mutation rate is sensitive to growth conditions, so cells grown in different conditions are likely to experience a different rate of Cit+ mutation.

:::::::Regardless of the reason for the lower-than-expected number of Cit+ mutants in the third experiment, however, the Cit+ mutants isolated absolutely did not arise from contamination. This is clear if you read the paper. You're still left, then, with the two hypotheses presented, and the results support historical contingency.[[User:Gerlach|Gerlach]] 09:45, 16 July 2008 (EDT)
Is there anyone here with expertise in statistics who could give an analysis? [[User:Fyezall|Fyezall]] 16:15, 15 July 2008 (EDT)

:Hello Fyezall. I rearranged the position of your question to hopefully keep the conversations clearer. It boils down to this: The researchers found that mutant Cit+ strains arose over the course of time in their long term growth experiment. They wanted to learn something about how that strain acquired this ability. They wondered, 'Was this the result of a single, very low frequency mutation or did some other mutation have to precede it in earlier generations, followed by the final mutation(s) that allowed the cells to grow on citrate?'

:If the Cit+ change required a single mutation with a low but constant probability over time one would expect Cit+ mutations to be distributed across cells taken from any generation of the experiment. On the other hand, if a 'potentiating' mutation had to arise at some point in the cultures before the final Cit+ mutation could function, then one would expect the probability for Cit- cells to mutate to Cit+ cells would increase with samples of cells taken from later cultures.

:Blount ran the experiment and found that Cit+ mutants arose more frequently in cells taken from later generations in the culture. The Monte Carlo resampling tests were used to assess how likely the pattern of results would fit the models. The statistical significance (smaller P-values mean greater significance), was calculated for each experiment and the combination of experiments. The numbers suggest the distribution of mutants was not randomly distributed across the experimental generations, they tended to appear in cells taken from the later generations. This would argue against the single-mutation, constant, low-probability hypothesis. It appears that a pre-adaptive mutation had to have arisen first, followed by the mutation that finally allowed the cells to utilized citrate. Future work in Lenski's lab will focus on trying to identify the various mutations involved. I hope this brief explanation helps.--[[User:Argon|Argon]] 20:25, 15 July 2008 (EDT)

==What I have noticed==
I tell you what strikes me the most about this discussion - Its watching evolutionists do whatever they can to keep their precious beliefs above question or scrutiny. Its as if they know that it will come toppling down so they must resort to side-lining the hard questions.
[[User:JJacob|JJacob]] 21:43, 15 July 2008 (EDT)
==[[Social Text]]==
This well known hoax ''does'' support Aschlalfy's position that the paper was reviewed by PNAC ''rather quickly'', since (apparently) the paper supported the editors/reviewers point of view. Perhaps this would have beeen better placed on the [[Richard Lenski]] page rather than here since this deals only with the 6/10/08 PNAS paper ''per se'' and not the review process. [[User talk:MargeryCampbell|Marge]] 12:45, 16 July 2008 (EDT)

:I fail to see the relevance of the Sokal Hoax here. You should also know that Social Text was not a peer-reviewed journal when Sokal submitted his paper. That doesn't excuse the credulity of its editors, but Sokal's paper was not sent out for review. Not to say that it would have mattered, postmodernism is just meaningless word salad.[[User:Gerlach|Gerlach]] 13:39, 16 July 2008 (EDT)

Talk:Flaws in Richard Lenski Study

2008-07-17T23:28:04Z

Argon: /* Formal Response to PNAS */

Now, is this page a report over other people thinking Lenski's paper is flawed or is this Aschafly reporting about himself and putting it in the headline of the Front Page? And please specify by references which "two other" experiments are referenced, and explaing how you can see that the "historical contingency" is not true. --[[User:Stitch75|Stitch75]] 23:22, 12 July 2008 (EDT)
And if you think you argue that well, why dont you submit it as a comment òn the paper to PNAS. --[[User:Stitch75|Stitch75]] 23:24, 12 July 2008 (EDT)

:This article is seriously partisan. I see no outside or third party analysis of the paper in the references, just a bunch of cites to the article or conservapedia itself. [[User:Wisdom89|Wisdom89]] 00:34, 13 July 2008 (EDT)

Folks, you're in the wrong place if you're more interested in who says what rather than determining the truth itself. A true wiki gets at the substantive truth rather than trying to rely on biased gatekeepers and filters of the truth.

The flaws in the statistical analysis in Lenski's paper are clearly set forth and well-referenced. If you're interested in the truth, then look at the paper and see the flaws yourself. If you're not interested in the truth and think you can distract people's attention from it by using other tactics, then you're wasting your time here.--[[User:Aschlafly|Aschlafly]] 00:42, 13 July 2008 (EDT)

I agree with Aschlafly on this, there needs to be some kind of admission or response from Lenski but little has been forthcoming and conservapedia itself has taken it on. Notice how no-one, aside from conservapedia (and I think Creationwiki?) has asked such questions of Lenski? All the magazines etc have taken his study at face value without actually taking the time to critique his claims. Aside from the "peer reviewers" of course.
[[User:JJacob|JJacob]] 00:47, 13 July 2008 (EDT)

: The "peer reviewers" who spent somewhere between 0 and only 14 days looking at the paper, and missed an obvious contradiction between Figure 3 (specifying the "Historical contingency" hypothesis) and Table 1, Third Experiment. The statistical analysis in the paper appears so shoddy to me that I doubt anyone with real statistical knowledge or expertise even reviewed it.--[[User:Aschlafly|Aschlafly]] 00:59, 13 July 2008 (EDT)
::I have expertise in research and statistics and I'm just not seeing this shoddiness that you make reference to. You are allowed to have your doubts, but we should get a bunch of people familiar with such fields to examine the paper's statistical analysis. [[User:Wisdom89|Wisdom89]] 01:01, 13 July 2008 (EDT)
::I have the same problem. i hold a Dr.Rer.Nat title and did some statistics (although i am no expert on it) and fail to see the "shoddiness" please help my underdeveloped mind, Mr. Schafly and enlighten me. I it is so obvious it should be a one-liner to formulate it. --[[User:Stitch75|Stitch75]] 09:54, 13 July 2008 (EDT)

::: Please try harder then. I've expanded the explanations a bit also.--[[User:Aschlafly|Aschlafly]] 10:42, 13 July 2008 (EDT)

:::: At least now i can recongnize your statements more or less clearly. Yet i think (cite from the paper) ''We also used the Z-transformation method (49) to combine the probabilities from our three experiments, and the result is extremely significant (P < 0.0001) whether or not the experiments are weighted by the number of independent Cit+ mutants observed in each one.'' has to be addressed more specifically than you do in order to discredit the statistics used --[[User:Stitch75|Stitch75]] 11:54, 13 July 2008 (EDT)
:::::Just for comparative purposes and a frame of reference, a P value that is less than the significance level of 0.05 is considered significant. [[User:Wisdom89|Wisdom89]] 13:10, 13 July 2008 (EDT)

I wonder, do the PNAS allow questions to raised and asked of the "peer reviewers" themselves? Are we able to find out who/what experience they themselves have? Perhaps that is an avenue that we could look at? I apologise in advance if this has already been asked or answered.
[[User:JJacob|JJacob]] 01:07, 13 July 2008 (EDT)

:Definitly not. Peer reviewers are anonymous and only known to the editor, for good reason. Having peer reviewers non-anonymous would cause reviewers to be very careful to step on nobodys foot to evade revenge. --[[User:Stitch75|Stitch75]] 09:54, 13 July 2008 (EDT)

: No, PNAS probably won't disclose who supposedly did the 14-days-or-less peer review on the [[Lenski]] paper. You're right that such disclosure could shed some light on the final product.

: "Wisdom89", your claim that you "have expertise" and don't see the flaws only makes me conclude that you don't really have the expertise that you claim. Judging by your silly user name, perhaps you've tried that approach before. We're not fooled by it here.--[[User:Aschlafly|Aschlafly]] 01:14, 13 July 2008 (EDT)

(removed silly comment by person who has since been blocked for [[90/10 talk]] violation)

==Marginally significant==
In #1, that's not inconsistent. The figure makes it clear that, according to the hypothesis, the mutations should also occur earlier than 31,000, but should become more common at that point. 12/17 mutants occurred after 31,000. The point in #3 is not clear. What do you mean by ''weighting''? In #5, ''Lenski's paper is not clear in explaining how the results of his largest experiment...his paper refers to his largest experiment as "marginally ... significant," which serves to obscure its statistical insignificance.'' Actually, '''marginally significant''' is clear. It means that the p-value is between .05 and .10 (in this case it's .08, table 2). It's a pretty standard phrase to describe an effect that falls into that range. [[User:Murray|Murray]] 13:41, 13 July 2008 (EDT)

: Your defense of Lenski per point #1 is contradicted by the paper's own abstract, and by the comments by the other Lenski defender below. In point #3, proper weighting is needed to combine multiple studies. In point #5, you don't cite any authority for the unscientific claim of being "marginally significant."--[[User:Aschlafly|Aschlafly]] 20:45, 13 July 2008 (EDT)
: Your defense of Lenski per point #1 is contradicted by the paper's own abstract, and by the comments by the other Lenski defender below. In point #3, proper weighting is needed to combine multiple studies. In point #5, you don't cite any authority for the unscientific claim of being "marginally significant."--[[User:Aschlafly|Aschlafly]] 20:45, 13 July 2008 (EDT)
::Define "proper weighting". Are you referring to how they interpreted the sum of the work or to a specific analysis? You're right, I didn't cite any scientific authority. Here's one: Motulsky, H. (1995). ''Intuitive Biostatistics'', Oxford Univ. Press. Chapter 12. Also, try searching for the phrase in Google Scholar or PubMed, you'll find plenty of uses of it. It's a shorthand way of describing an effect that came close to the arbitrary threshold for statistical significance but did not reach it. [[User:Murray|Murray]] 21:55, 13 July 2008 (EDT)

::: You seem unwilling to accept that Point #1 identifies a clear mistake between the figure and the abstract in the paper, which requires correction by Lenski or PNAS. Given that unwillingness, I doubt it will productive discussing the other mistakes further with you. You're right that other usage of the dubious concept of "marginally significant" can be found on the internet, but the first link to such usage on my internet search for it returned the non-rigorous "Intuitive Biostatistics" and the second link returned a criticism of the concept similar to the criticism express here. [http://annhyg.oxfordjournals.org/cgi/reprint/49/1/93-a.pdf] --[[User:Aschlafly|Aschlafly]] 00:42, 14 July 2008 (EDT)
::::It's not a clear mistake in my mind. There does seem to be some confusion or contradiction in terms of the number of generations, but I haven't seen a clear explanation of why this is or is not a mistake. What seems like a contradiction is the abstract statement that no mutations occurred before 31,500, but it's not clear to me whether I'm misunderstanding that. ''I doubt it will productive discussing the other mistakes further with you. '' Of course you doubt it, because you are unlikely to be willing to concede anything no matter what anyone says. Why do you call the concept dubious? The procedure of determining whether an effect is significant requires the setting of an arbitrary threshold, which is usually .05. That means, in analyses of the sort in the Blount et al. paper, that there's less than a 5% chance that the findings are due to sampling error. When an effect comes close to the threshold it is worth noting, because of the problems inherent in significance testing, which itself is widely criticized in the statistical literature. I am not clear what you mean by weighting, as I mentioned before - in the interpretation or statistically? [[User:Murray|Murray]] 13:30, 14 July 2008 (EDT)

== Richard Lenski incorrectly included generations of the E. coli already known to contain Cit+ variants in his experiments.[3] Once these generations are removed from the analysis, the data disprove Len ==

The author of the above statement must not have read the paper or the supplement carefully. Lenski took clones from those cultures that '''weren't Cit+'''. Careful reading of the paper or the supplement reveals that Cit+ mutants appeared at 750 generations or later into the replay experiment. The authors write: "New Cit+ variants emerged between 750 and 3,700 generations..."--[[User:Argon|Argon]] 15:03, 13 July 2008 (EDT)

: Your explanation is not convincing. Show us how Lenski proved that the samples did not have '''any''' Cit+ variants, if you really think he's claiming that.--[[User:Aschlafly|Aschlafly]] 20:51, 13 July 2008 (EDT)

::I count at least four distinct diagnostic methods used in the paper for distinguishing whether clones are Cit+. One in particular they describe as being very sensitive to weakly citrate-using cells. It's not that I 'think' Blount et al. claim they started with Cit- cells. They say that in the 'Supporting information' document.

::Andy, for tonight, I'll leave it as an exercise for you to answer your own question of how one might test a clone to be sure that is was Cit-. Use the paper if you'd like or present another means. Tomorrow evening I'll cite the methods Blount used in the paper.

::Meanwhile, here's another question to ponder: If the colonies used to start the 'replay' cultures were Cit+ at the start, why is it that no Cit+ cells were found in such cultures before 750 generations? These cultures were all started from single clones which must have either had the Cit+ phenotype or not.--[[User:Argon|Argon]] 18:38, 14 July 2008 (EDT)

-
How did Lenski obtain Cit- clones from later populations after Cit+ had become the dominant phenotype? It's pretty standard procedure. First, keep in mind that these populations contained a minority of Cit- cells. Now, a small liquid growth is made from the frozen glycerol stock of cells from a given generation, say 32,000. This is grown to an appropriate OD and then plated on a standard rich media agar plate. Provided that the plated sample isn't too dense, this plating will deposit somewhere between 15-100 individual cells on the agar, with plenty of spacing between them. The plate is then incubated at 37 C for a given time, during which the individual cells replicate to form small, visible colonies. These colonies will consist solely of cells that are genetically identical to the original cell deposited on the agar. This is all well and good, but how do you find which of the colonies are Cit-? Here you use a technique called replica plating. The agar plate is gently inverted onto a sterile swatch of velvet so that some of the cells from each colony are deposited onto the fabric. Next, a citrate-only agar plate is pressed against the velvet, transferring cells from the fabric to the new plate in exactly the same spacial orientation as the first plate. The cells are then allowed to grow on the citrate-only plate. When we then compare the two plates, we look for colonies on the rich plate that did not grown on the citrate-only plate. These colonies will be Cit-, and can be used for the replay experiments. [[User:Gerlach|Gerlach]] 13:45, 14 July 2008 (EDT)

:Just a correction: The Cit+ cells didn't dominate the cultures at generations 32,000 & 32,500. Cit+ mutants represented 12% and 19% of the population at these times points, respectively. Replica plating is definitely one means of identifying Cit- and Cit+ clones but because the Cit+ cells represented a fraction of the total, it is simple enough to streak the samples to individual colonies (founded by single cells) and test each colony individually. Otherwise, a nice description of replica plating (a technique developed by Joshua Lederberg that allowed him do the research for which he was awarded one of the 1958 Nobel Prizes in Medicine)--[[User:Argon|Argon]] 18:38, 14 July 2008 (EDT)

::Oops, you're right, Argon. Cit+ wasn't dominant at generation 32,000, so replica plating might not be the best option. Still, in the case of almost 20% Cit+, I think replica plating might be labor reducing. I'm a biochemist by training, not a microbiologist, but I'm sure there are other methods that could be used. For example, Christensen's agar appears to provide a sensitive, colorimetric method of identifying even weakly citrate-utilizing colonies, so one might be able to plate cells on a Christensen's agar plate and pick uncolored colonies. Again though, I'm no microbiologist so I don't know the best method. However, there are ways to easily pick Cit- clones from later generations in this case. That said, I think that this specific "flaw" cited in Lenski's paper should be removed from the main article. Clearly, Cit+ were not used in the replays from later generations, so Conservapedia's objection is totally baseless.[[User:Gerlach|Gerlach]] 19:19, 14 July 2008 (EDT)
:::All appropriate descriptions. For generating estimates of the population distribution (Cit-/Cit+) in a culture, replica plating would certainly be applicable. The main point is that identifying and isolating Cit- clones from the generations used in the experiment is straightforward. I agree that it would be a mistake to keep that objection in the article (actually that thread runs across several points in the article).--[[User:Argon|Argon]] 20:12, 14 July 2008 (EDT)

:::: Nothing in the paper rules out contamination of those samples by Cit+ variants, and you quote nothing in the paper to rule it out. There was no reason to use and rely on these samples that already have Cit+ variants.--[[User:Aschlafly|Aschlafly]] 23:04, 14 July 2008 (EDT)
:::::The whole point of the discussion above was to illustrate that there are methods to isolate cells with a particular genotype from a mixed population. Suppose we used the replica plating method (maybe not the best choice, but it's standard procedure). Care to explain where the Cit+ contaminants could possibly come from? And the naivete inherent in objecting that "nothing in the paper" rules it out is simply astounding. Standard laboratory procedures, like isolating specific clones, typically don't show up in papers. For example, if I to write a paper on my research, it would not include a detailed discussion of the construction of my expression plasmid. I might mention that I cloned my gene into a particular vector, but there wouldn't a discussion of how I did my PCR, restriction digests, or transformations. Things like that are simply extraneous details that are taken for granted by experienced researchers. There is simply no reason for Lenski et al. to include a discussion of their method for isolating specific clones. This is why this whole exercise of Conservapedia criticizing Lenski's paper is folly, because most Conservapedia users simply don't know standard laboratory techniques. It's kind of important to know what scientists today are able to do on a routine basis before wading in to claim that they couldn't have done what they claim.[[User:Gerlach|Gerlach]] 09:05, 15 July 2008 (EDT)

:::::***** *****
:::::Actually the methods cited in the paper would separate the Cit+ lines from the Cit-. They took single colonies (founded by single cells), and tested them on selective agar (minimal citrate media - MC agar) and an indicator medium, Christensen's citrate agar (Product information from Sigma here: http://www.sigmaaldrich.com/sigma/datasheet/c7595dat.pdf). As Gerlach notes from the Blount paper, Christensen's citrate agar is very sensitive to citrate utilization. Typical ''E. coli'' strains do not produce a color change but other citrate-using enterics like ''S. typhimurium'' and Cit+ ''E. coli'' mutants appear pink/red on the plates.

:::::This conclusion is further validated (as I mention above), by the fact that Cit+ mutants did not appear immediately in the replay experiments (Others have also noted this). For example, in replay set-1, it took 750 generations before the first Cit+ mutants were isolated. Many didn't produce Cit+ cells after 3700 generations. If the starting line was Cit+, *all* the cells in the culture would have show up as Cit+ in the first pass.

:::::Overall, it is abundantly clear that the cell lines used in the replay experiments were not Cit+ at the beginning.--[[User:Argon|Argon]] 19:44, 15 July 2008 (EDT)

== 1. Lenski's "historical contingency" hypothesis, as specifically depicted in Figure 3, is contradicted by the data presented... ==

Again, more issues with comprehension: The authors did not know when the potentiating mutation first arose but they knew it was before generation 31,500. Figure 2 is merely figurative, being an illustration, and not quantitative. Their analyses suggested that the potentiating mutation did arise at about the 20,000 generation point or later. Their conclusion is that Cit+ mutation rate is low even in a potentiated background but apparently distinguishable from a low-incidence single, unpotentiated event.--[[User:Argon|Argon]] 15:04, 13 July 2008 (EDT)

== Statistics101 package: ==

What does the author imply by writing: "...Lenski himself does not have any obvious expertise in statistics. In fact, Richard Lenski admits in his paper that he based his statistical conclusions on use of a website called "statistics101"?

From the web site:
''Professionals: Although it was originally developed to aid students, the Statistics101 program is suitable for all levels of statistical sophistication. It is especially useful for Monte Carlo, resampling, and bootstrap applications. It has been used by professionals in many fields. These include anthropology, biology, ecology, evolutionary biology, epidemiology, marine biology, psychology, toxicology, veterinary pathology.''
It would appear that the package from the web site (not the web site itself) was used to perform the Monte Carlo resampling tests. Is there any evidence that the package produces incorrect results?--[[User:Argon|Argon]] 19:41, 13 July 2008 (EDT)

==Formal Response to PNAS==

''I've moved this thread from the open section above since this page is being categorized.'' --[[User:DinsdaleP|DinsdaleP]] 21:52, 13 July 2008 (EDT)

:::I think this article presents very sound arguments. Conservapedia should now take action, offering to publish a rebuttal of Lenski in the PNAS journal.--[[User:JBoley|JBoley]] 11:31, 13 July 2008 (EDT)

I agree with JBoley in that if Conservapedia wants to present a formal, professional response to Professor Lenski's paper that questions specifics within his paper, then it should happen. That is the proper execution of the scientific method, and I'm certain that a professional response to PNAS would yield better results than vague "give us all the data " demands. Is a formal response to PNAS from Conservapedia in the works, or is this article the only place these questions/objections were intended to be raised? --[[User:DinsdaleP|DinsdaleP]] 11:52, 13 July 2008 (EDT)

: I don't know if PNAS would embarrass itself by printing a rebuttal, or whether it has the integrity to retract Lenski's paper. Conservapedia's audience is probably bigger than PNAS's, and we're certainly not going to suspend our exposure of the truth here in order to await correction by PNAS.

: PNAS publishes, quite easy to find Rules for "Rebuttals" (they call it "Letter"). It is specifically what you want. [http://www.pnas.org/site/misc/iforc.shtml PNAS:Information for Authors] . I cite: ''Letters are brief online-only comments that contribute to the discussion of a PNAS research article published within the last 3 months. Letters may not include requests to cite the letter writer's work, accusations of misconduct, or personal comments to an author. Letters are limited to 250 words and no more than five references.'' --[[User:Stitch75|Stitch75]] 13:25, 14 July 2008 (EDT)

: In addition, Lenski has already demonstrated how he reads this site and he can certainly correct his own paper, and he should do so. Indeed, professionalism might support giving Lenski the time to correct it himself first.--[[User:Aschlafly|Aschlafly]] 12:14, 13 July 2008 (EDT)

::I understand what you're saying, but there's nothing improper or unprofessional in submitting a formal request to PNAS to have the points in this article addressed by Professor Lenski and his team. To be frank, you've been adamant in your insistence that PNAS has been less than rigorous in the review of Lenski's paper, so if one of your intentions is to demonstrate this then having PNAS respond to a formally submitted response to the paper in public would serve that purpose. This can be done in addition to publishing these objections on Conservapedia--[[User:DinsdaleP|DinsdaleP]] 12:21, 13 July 2008 (EDT)

:::If not PNAS, then perhaps some other public forum? I know Andy Schlafly has appeared on television, effectively arguing against gardisal and other dangerous vaccines. Perhaps if a TV program were interested you could argue against Lenski? You could be the spokesperson against these false claims of evolution.--[[User:JBoley|JBoley]] 12:24, 13 July 2008 (EDT)

:::: I'm not opposed to the above suggestions, but the future is here, folks. Lenski, PNAS editors and television producers have free will to reject or ignore the truth, and I'm more interested in getting the truth out here than trying to persuade someone in dying media like print or television. Lenski and his defenders can see the truth here, and they can decide for themselves whether to reject or admit it.--[[User:Aschlafly|Aschlafly]] 12:30, 13 July 2008 (EDT)

:::::I agree with you about the dying nature of print (I don't think television is dying, merely changing). The problem is that information about the flaws in Lenski's study are not registering outside of sites like Conservapedia. In effect, Conservapedia is an echo chamber. People that come to this site already agree with its point of view. I encourage you to attempt to attract the attention of other forms of media, or Lenski's false claims will simply be accepted as fact by the public and even worse, by educators.--[[User:JBoley|JBoley]] 12:36, 13 July 2008 (EDT)

::::::While there's nothing wrong with taking one's message to various forums or outlets, I believe there's a specific value in submitting these objections as a formal response to PNAS. Conservapedia was established as a trustworthy resource for students, and in my mind all of it's actions should be done with the goal of informing and educating. The Lenski debate is over the findings published in a scientific journal after undergoing a peer-review process. The objections to this paper by the CP leadership are not just about its content, but to the process by which it was reviewed and published in the timeframe it was. Talking about these objections is fine, but it's more instructional to the students using Conservapedia, and a better example of the scientific method in action, to respond to a scientific paper published in a journal through the formal process by which such papers are either defended or corrected. In the end, Lenksi's work will either stand up as good science, or any errors will be addressed and the paper's conclusions modified accordingly, which is also good science. Seeing this process in action regarding a such a significant paper is a great learning opportunity, and the Conservapedia leadership would be remiss in not standing by their conviction in these objections and submitting them formally to PNAS. --[[User:DinsdaleP|DinsdaleP]] 12:38, 13 July 2008 (EDT)

PNAS has a letters section available with the online edition. Many journals have a letters section for rebuttals or clarification. Legitimate corrections are welcome. Andy, have you run your list of 'flaws' past any biologists?--[[User:Argon|Argon]] 15:36, 13 July 2008 (EDT)

: Funny, Argon, how you don't apply your demand of expertise to Lenski himself. What are Lenski's credentials with respect to statistical analysis? Has he even taken and passed an upper-class statistics course of any substance?--[[User:Aschlafly|Aschlafly]] 00:45, 14 July 2008 (EDT)

::I don't follow. Are you suggesting that Lenski doesn't understand the proper application of the statistical methods used in his paper? If so, I haven't seen a description of which alternate methods you'd employ, let alone any output from such an analysis. Here's a thought: Why don't you substantiate your claims by writing up the work and submitting it as a correction letter to PNAS?--[[User:Argon|Argon]] 18:46, 14 July 2008 (EDT)

::: Argon, if you really "don't follow," then try harder. You insist on credentials by others who criticize Lenski, and yet you do not insist on expertise by Lenski in statistics with respect to his "analysis". Perhaps Lenski should first take and try to pass "Statistics 101" before trying to use a website by its name to draw flawed conclusions.--[[User:Aschlafly|Aschlafly]] 22:36, 14 July 2008 (EDT)

::::Do you really think that Lenski went to statistics101.net to learn about statistics and how to apply them? That simply is not the case. The Lenski group knew that they needed to do a Monte Carlo resampling analysis on the results of their replay experiments. In this situation, they are faced with two choices: either code an appropriate program themselves, or utilize one that has already been developed and is readily available to researchers. Since statistics101 had such a program available, they chose the latter option. Statistics101.net was simply the source of the program that the Lenski group used to perform the statistical analysis. If you want to argue against that choice, then you need to examine the source code for the statistics101 package and enumerate why it should not have been used.[[User:Gerlach|Gerlach]] 11:40, 15 July 2008 (EDT)

::::***** *****
::::I don't insist on credentials per se. I *recommend* developing a working understanding of the experiments or a willingness to do the necessary background research to get the details straight before heading off on possibly the wrong direction. It saves a lot of thrashing about. It's perfectly OK to raise questions but before leveling accusations it might be nice to do that privately and discuss that with others who can provide useful feedback. Just my 2 cents.--[[User:Argon|Argon]] 19:48, 15 July 2008 (EDT)

::Mr. Schlafly, does your response above, "I'm more interested in getting the truth out here than trying to persuade someone in dying media like print or television", mean that you will not be submitting a response with these identified flaws directly to PNAS? As I mentioned above, it takes nothing away from the value of posting these statements here on CP to also submit them to PNAS, but the proper way to prompt a journal to review and correct an article is through a direct response, not publication on an unrelated website like CP. It's not proper for anyone but the author(s) of the objections in this article to make that submission, so I'm hoping they step up with the conviction of their beliefs and respond to PNAS directly. Thanks. --[[User:DinsdaleP|DinsdaleP]] 12:32, 14 July 2008 (EDT)

::: By now Lenski has probably seen the flaws identified on the content page here. What's "proper" is for him to correct his own paper in PNAS. The criticism will likely continue as long as he declines to do so. If anyone here would like to educate the PNAS editors about the flaws, then please feel free to do so.--[[User:Aschlafly|Aschlafly]] 22:36, 14 July 2008 (EDT)

::::I'll be glad to submit these items to PNAS on behalf of Conservapedia using the "Letters" forum described by Stitch75 above. Should I cite you the author of this analysis, and is there an email you'd prefer me to include instead of my personal one for any PNAS response? I'll put up a draft of the Letters submission here for your approval before sending anything out. Thanks. --[[User:DinsdaleP|DinsdaleP]] 18:39, 15 July 2008 (EDT)

:::::It would have a very strange scientific taste should somebody else that Mr. Schlafly be the first author of the letter (however, if you insist on that, i suggest "personal communications" as the right kind of citing the work here.). It is his turn to stand up to his claims in this way (by publishing it into a Forum which he is not the owner of). I personally share the doubts of many here about his argumentation (and his understanding os the experiments), but i am sure only more of his dismissive comments will follow. However i see that he comes up with a clear alternate Hypothesis (contamination), so he is free to show that this is more likely (calculations please) and peform Monte-Carlo simulations on it (for a person complaining that statistics101 is to simple that should be no difficult task). As far as i understood and see the data Lenski and coworkers did the best to exclude this Hypothesis, however i did not run own simulations (And I won't do it, because i think nothing will come out - furthermore Mr. Schlaflys personal style in the communication "you have to try harder" is not the style i am used to be adressed by people whose qualification in a subject is appearlingly nor more than mine). So running the simulation, evaluating his own hypothesis using a valid statistical method is now Mr. Schlaflys job - if he come up with a decent calculation showing this Hypothesis is more likely, the letter would for sure be accepted and Lenksi would have to react. If Mr. Schlafly is not the first author of the letter he could evade the critics after that by saying that he was misunderstood, which means somebody else take the risk of submitting the letter, but in case of success Mr. Schlafly would take the glory. --[[User:Stitch75|Stitch75]] 20:10, 15 July 2008 (EDT)

::::::I would prefer that Mr. Schlafly submit his objections to PNAS directly as well, but since he has declined to do so the next best response is to submit it "on behalf of Conservapedia", which he has authorized above. I'll post the draft letter tomorrow, and it will credit him as the author unless I'm asked to include other individuals who contributed to the analysis. --[[User:DinsdaleP|DinsdaleP]] 22:05, 15 July 2008 (EDT)

:::::::I hope ASchlafly does offer a rebuttal of Lenski's flawed "study." However, if he does not, I am all for DinsdaleP's suggestion. I look forward to reading your draft. If you take all the objections that Conservapedians have raised to Lenski's paper, I do not see how PNAS can possibly object.--[[User:JBoley|JBoley]] 11:28, 16 July 2008 (EDT)

::::::::They could object because Conservapedia's criticisms are obviously false. If this page hasn't made that clear, I don't know what could.[[User:Gerlach|Gerlach]] 13:44, 16 July 2008 (EDT)

:::::::::They would have to prove that Conservapedia's criticisms are false, and they cannot do that. You sound like a Lenski supporter. You need to open your mind to the truth.--[[User:JBoley|JBoley]] 13:59, 16 July 2008 (EDT)

:::::::::Gerlach, the most obvious lesson from this page is how a few, yourself included, seem determined to defend a flawed paper no matter what the truth brings. You have free will to reject whatever you want, but you're only hurting yourself by that approach. People who do open their minds are amazed by the insights and happiness it brings.--[[User:Aschlafly|Aschlafly]] 21:54, 16 July 2008 (EDT)

:::::::::This is such a strange response, Andy. The attitude that you and several of your defenders seem to have is that the criticisms you have made of Lenski's paper cannot be rebutted. However, the fact that you ''can'' level criticism at something doesn't make that criticism true. That is, you may be wrong. And that is the case here. Myself and others have pointed out that your alleged flaws in Lenski's paper are incorrect, and based on misunderstanding it, or, even worse, simply not reading it carefully. There has been little to no substantive response to our detailed rebuttals. Replies from you or other defenders of the Conservapedia article amount to little more than brazen declarations that we are wrong or "nonresponsive", and that criticisms in the original article remain unscathed. But for people who claim to have "the truth" on their side, this is simply baffling. If the veracity of your claims against Lenski's paper is so obvious, then it should be an easy task to provide in depth responses to the points that we have been raising against your article. I'm not above criticizing scientific papers. Bad papers get published frequently, after all. However, Lenski's paper does not appear to be one of those. Notice, though, that I am not claiming it to be perfect, no paper is. That said, whatever flaws the paper may have, those presented in the Conservapedia article are not among them. [[User:Gerlach|Gerlach]] 11:43, 17 July 2008 (EDT)

:::::::::: Gerlach, now you seem to admit that the Lenski paper may be flawed, but that Conservapedia has not identified any of those flaws! With all due respect, you seem to have taken closemindedness to new heights.

:::::::::: Lenski claims there is a mutation rate, yet his presented data show that the number of mutations do not scale with sample size. His presented data disprove both of his hypotheses.--[[User:Aschlafly|Aschlafly]] 13:27, 17 July 2008 (EDT)

:::::::::: First, we need to clarify what we mean by "flaw". Are we talking about serious methodological flaws or errors in interpretation that fatally undermine the paper's conclusion? If so, I fail to see any such flaws. Of course, any honest scientist recognizes that any work at the frontier of science has the potential to be wrong. It ''may'' be that future work shows Lenski to be wrong. But let's not confuse this possibility with actuality: saying that something might be wrong is not the same as saying that it is. On the other hand, by "flaw" we may mean minor experimental details that could have been better. In this case, the paper is flawed, and Lenski himself admits this in the supporting information. But ''every'' paper is flawed in this manner, and I doubt you'll find any investigator who wouldn't say that they wish they had done some things differently during the course of their research. As an example of what I'm talking about, the Lenski group's statistical analysis would have been improved if they could have accounted for the evolution of increased cell size (and, therefore, decreased cell density) in later generations. As it stands, their analysis ''underestimates'' the potentiation effect in these generations because replays of later generations involved fewer cells. But this flaw, and other flaws of this type, do not undermine the conclusion of the paper.

:::::::::: Regarding the mutation rate argument, see the earlier discussion on this page. The third experiment was significantly different than the first two. The experimental scheme was different, it was performed at at different time and with different conditions, and utilized, for the most part, different clones than the first two. And Lenski doesn't just claim there is a mutation rate in potentiated cells, he ''actually measures it''. But frankly, I don't understand your argumentation here. It seems that you are trying to suggest that all the Cit+ mutants isolated were the result of contamination. This clearly isn't the case, for several reasons. However, this isn't the place for this particular discussion, as there is already such a discussion elsewhere on this page. I still haven't seen any adequate response to our points against the Conservapedia article, and I don't think I am closed-minded for expecting such a response. As I said, I'm open to the possibility that the Lenski paper is flawed, but I expect cogent argumentation to support any such flaws. I haven't seen that here.[[User:Gerlach|Gerlach]] 15:09, 17 July 2008 (EDT)

::::::::::: Gerlach, look at how many words you used to sidestep my simple explanation of a flaw in Lenski's work: "Lenski claims there is a mutation rate, yet his presented data show that the number of mutations do not scale with sample size. His presented data disprove both of his hypotheses." If the third experiment of Lenski's was independently flawed as implied by your response, then that does not help your defense. Note that in Lenski's second experiment the mutations also failed to scale with sample size.--[[User:Aschlafly|Aschlafly]] 18:26, 17 July 2008 (EDT)

::::::::::::I think there is a misunderstanding. Blount et al. did not say that the absolute mutation rate was constant for the generation of Cit+. After all, they found that variations in culture conditions had some effect. The replay experiment was performed to determine if clones from later cultures were more or less likely to give rise to Cit+ cells than clones from earlier generations. The authors' hypothesis was that a potentiating mutation was required before the Cit+ mutations could arise. If that was the case then clones from later generations would be more likely to produce Cit+ cells. If the Cit+ capability was the result of an extremely rare, single mutation, then any generation of clones would be equally likely to produce Cit+ cell. The hypothesis is that in a particular experiment, the '''relative probability''' of generating a Cit+ mutant would be greater with clones from later generations. '''Absolute mutation rates''' (which appear to be Andy's concern) may be contingent on the growth conditions, which differed between the three replay experiments but '''''within any particular set of conditions''''', one might expect the relationships between Cit+ recovery and the generations from which the starting clones were derived would still hold.

::::::::::::pg 7902 of the paper in the journal:
::::::::::::"According to the rare-mutation hypothesis, Cit+ variants should evolve at the same low rate regardless of the generation of origin of the clone with which a replay started. By contrast, the historical-contingency hypothesis predicts that the mutation rate to Cit+ should increase after some potentiating genetic background has evolved. Thus, Cit+ variants should re-evolve more often in the replays using clones sampled from later generations of the Ara-3 population."

::::::::::::From the abstract:
::::::::::::"The long-delayed and unique evolution of this function might indicate the involvement of some extremely rare mutation. Alternately, it may involve an ordinary mutation, but one whose physical occurrence or phenotypic expression is contingent on prior mutations in that population. We tested these hypotheses in experiments that ‘‘replayed’’ evolution from different points in that population’s history. We observed no Cit+ mutants among 8.4 x 10^12 ancestral cells, nor among 9 x 10^12 cells from 60 clones sampled in the first 15,000 generations. However, we observed a significantly greater tendency for later clones to evolve Cit+, indicating that some potentiating mutation arose by 20,000 generations. This potentiating change increased the mutation rate to Cit+ but did not cause generalized hypermutability."

::::::::::::What the authors found was that the later generations really did produce more Cit+ mutants than you'd expect if the Cit+ mutation was instead randomly distributed (or, as Andy has claimed and not yets retracted, contaminated by Cit+ cells). Yes, the absolute rates didn't scale across the three difference conditions but within each experiment it is clear that the Cit+ mutants arose from cells taken at later generations. Yes, there were differences in the absolute rates under different conditions but that doesn't mean the results and conclusions about potentiated clones are wrong.

::::::::::::As for calculations of mutation rates: Blount et al. performed additional fluctuation experiments (journal pg. 7903) in an attempt to estimate the relative effect of the potentiating mutation and calculate a rough estimate of the mutation rates. Keep in mind, those rates '''''are referenced to growth under the specific conditions used in that particular experiment''''' and were used to provide ballpark estimates for comparison to other classes of known mutations. As for the second experiment not 'scaling' (with the first?), I wouldn't expect that. The first involved clones grown in continuous liquid subcultures and both the total number of generations, cells/generation and growth conditions (e.g. liquid with nutrient replenishment vs. solid agar) are very different.--[[User:Argon|Argon]] 19:20, 17 July 2008 (EDT)

:::::::: In response to earlier postings about drafting and submitting a letter to PNAS, I'm for it and would be happy to contribute. In response to a comment above, I don't want any "glory" and learned a long time ago that nobody gets credit or money for telling the truth. More often those who speak the truth are reviled and insulted, but mockery doesn't bother the truth as much as it bothers falsehoods.--[[User:Aschlafly|Aschlafly]] 22:01, 16 July 2008 (EDT)

:::::::::Although I disagree with most of Andy's conclusions, I am in favor of Andy signing and submitting his comments to the PNAS on behalf of Conservapedia (and the Eagle Forum , assuming they provide some sponsorship of this site). That is, after all, within his rights and possibly a responsibility as well. Given that Andy outlined most of the pages on the subject and authored most of the core arguments on the pages, I think it would be crazy for Andy to walk away from what has clearly been his leadership role.--[[User:Argon|Argon]] 19:27, 17 July 2008 (EDT)

== Expertise in Statistics ==
I don’t mind admitting that I have none! The article states that Lenski combined the three trials incorrectly and that he doesn’t make the insignificance of the third clear. Now it may well be very clear to others but, unfortunately, the article doesn’t make clear to people like me what was wrong. Can someone expand on this point? Is anyone able to explain to a layman what Lenski should have done and the conclusions he should have reached? It's all getting a little technical for me. --[[User:Billd|Bill Dean]] 12:10, 14 July 2008 (EDT)

: OK, here's point 4 in layman's terms: Lenski's hypotheses of a mutation rate imply that a ten-fold increase in sample size should result in a ten-fold increase in mutations. But it doesn't. In fact, a nearly ten-fold increase in sample size results in only a slight increase in mutations in Lenski's data. These data, as presented by Lenski in his paper, suggest (if properly interpreted) that there is no mutation rate at all. Rather, these data are more consistent with occasional contamination, broadly defined.--[[User:Aschlafly|Aschlafly]] 23:15, 14 July 2008 (EDT)

::Except that the growth pattern of the Cit+ cells in the third replay experiment demonstrated that there were no Cit+ cells in the cultures at the time of plating. Control plates with a mix of Cit- and Cit+ cells were done which showed rapid development of Cit+ colonies, whereas the experimental plates did not show any Cit+ colonies for at least 8 days and up to 28 days. If you believe that the plates became randomly contaminated during the course of the incubation, rather than at the initial plating, the post-plating contamination should have affected all generations equally, but it did not--[[User:Brossa|Brossa]] 12:46, 15 July 2008 (EDT)

::Blount et al. discussed the unexpectedly low Cit+ conversion rate for the third replay set in the supplementary document. They have some speculations but don't know why it occurred. They are dealing with low probability events. I don't agree that the 'proper' interpretation is contamination. First, they isolated and tested each Cit+ isolate for the markers and sequences we've discussed on earlier Conservapedia pages. Second, the distribution of Cit+ isolates does not appear to be random: They correlate strongly with the later generations. If undetected contamination was random, it's unlikely the Monte Carlo resampling tests would reject the null hypothesis of the 'rare-mutation' hypothesis over the 'potentiated' hypothesis. As Brossa correctly notes, contamination would have affected all generations. Blount et al. write in the paper's supplement: "To facilitate handling and minimize possible confounding variables, we divided this third experiment into 20 blocks of 14 clones each. All of the clones within a block came from different generations, and the single ancestral clone was included in all 20 blocks."--[[User:Argon|Argon]] 20:01, 15 July 2008 (EDT)
::: Good and detailed analyis and conclusion, Argon. I had no time to read everything in that detail, but i think you present exactly the arguments (contant background would favour rare mutation hypothesis) would be exactly the one i had in mind, so let's see how Mr. Schlafly will bend his own Monte-Carlo simulations to showing several 1000 times no mutation into accordance with any reasonable background explaining a significant number of counts in *some* samples --[[User:Stitch75|Stitch75]] 20:20, 15 July 2008 (EDT)

:::: The above comments are non-responsive. The essence of ''both'' of Lenski's hypotheses is that there is a fixed, or stepped, mutation rate. But any such rate would be roughly proportional with sample size. Yet his three experiments prove otherwise, which Lenski fails to address in a satisfactory manner. Indeed, Lenski's presentation of his data disproves the very thing he claims to have shown.--[[User:Aschlafly|Aschlafly]] 20:40, 15 July 2008 (EDT)

::::: Mr. Schlafly, please answer: Which the of the two hypotheses is, according to your calculations the most likely? --[[User:Stitch75|Stitch75]] 20:47, 15 July 2008 (EDT)

::::::The first replay experiment was run under different conditions - roughly 3700 generations in continuous subcultures. One can't compare population sizes or rates between the first and last two replay experiments. As I noted, Blount et al. acknowledged the possible anomaly of lower than expected mutants in the third run. Still, the second and third experiments were run under different conditions. The plates in the second replay experiment were seeded with fewer cells per plate than in the third experiment. That change can affect the survival rates of cells on plates over time (e.g. different rates of nutrient exhaustion). Given the extremely low mutation rates involved, there is no simple means of normalizing the numbers of mutants recovered for the second and third experiments. One might expect 'roughly' 10x more mutants in the third experiment but that's truly a 'rough' estimate that would be affected by conditions under which cells are exposed. It's certainly a question that remains and they may be able to clear up with future research. In any case, the pattern of data does not support Andy's claims of contamination or that the hypotheses of the paper is in error.--[[User:Argon|Argon]] 21:27, 15 July 2008 (EDT)

::::::: Argon, you have free will and it's clear to me that you are going to exercise it to the point of embracing absurdities. This time you claim that "extremely low mutation rates" would not result in mutations that scale with sample size (of course they would), and that density completely alters mutation rate (if that silly claim were true, then Lenski's experiment was flawed from the get-go). Your belief system is remarkable, but it's not logical.--[[User:Aschlafly|Aschlafly]] 23:38, 15 July 2008 (EDT)

::::::::Fair enough, but let's be clear: That's my take on the paper and that of most of the scientists who reviewed the paper, read the paper and commented upon the paper so far. For that matter, Michael Behe didn't call the data flawed, nor did those commentators at Dembski's Uncommon Dissent blog, nor did Dr. Georgia Purdom at Answers in Genesis. In contast, those who think the work is flawed appears to be limited mostly to you, Andy. It is true that 'scaling' was not seen in this case but as we've seen, the conditions were not quite the same and it is known that this can have an impact.

::::::::However, the fact that the conditions were not identical doesn't detract from the fact that the emergence of Cit+ clones *still* correlated with the sampling of the later generations. What this means is that with three separate experiments and under three sets of conditions, the constant-rate-mutation hypothesis doesn't hold. What the differences between the second and third replay experiments demonstrates is that they were run under different conditions that affected the overall rate of conversion, not that the 'potentiated mutation' hypothesis is wrong. Those are actually distinct questions. Andy, the data simply does not support your claims that the cultures were contaminated (we'd expect random distribution), or that the 'scaling' variations ruined the experiment. In my opinion, you seem focused on red herrings to the exclusion of evaluating the data in the overall context of the experiment which demonstrates a correlation of Cit+ clones emerging from samples taken at later generations. Would you care to address that pattern and discuss why your 'contamination hypothesis' doesn't appear to hold up?--[[User:Argon|Argon]] 09:29, 16 July 2008 (EDT)

:::::::There is nothing absurd in what Argon said. As he mentioned, the third experiment was performed under different conditions and at a different time than the first or second experiments. The key here is that there are plenty of variables, some unknown, that simply aren't controlled for between the experiments. For example, take the fact that it was performed at a different point in time than the first two. The third experiment, then, is almost certainly being performed with different batches of growth media, liquid and solid. Anyone who has spent any significant time growing cells knows that media can vary significantly in growth characteristics between batches, despite the same recipe being used. The reasons for this can be many. Perhaps the balance or pH meter was off calibration one day, or a different bottle of reagent was used. Take an analytical chemistry course, you'll spend plenty of time talking about this. It is true that, for the most part, this sort of variability has minimal impact. But when you're examining something like an extremely rare mutation, or you're trying to make an extremely accurate measurement, intra-lab variability like this can be significant. For an extremely rare mutation like Cit+, which involved at least two additional mutations in potentiated cells, any change in mutation rate can have a significant effect on your ability to obtain mutants. Mutation rate is sensitive to growth conditions, so cells grown in different conditions are likely to experience a different rate of Cit+ mutation.

:::::::Regardless of the reason for the lower-than-expected number of Cit+ mutants in the third experiment, however, the Cit+ mutants isolated absolutely did not arise from contamination. This is clear if you read the paper. You're still left, then, with the two hypotheses presented, and the results support historical contingency.[[User:Gerlach|Gerlach]] 09:45, 16 July 2008 (EDT)
Is there anyone here with expertise in statistics who could give an analysis? [[User:Fyezall|Fyezall]] 16:15, 15 July 2008 (EDT)

:Hello Fyezall. I rearranged the position of your question to hopefully keep the conversations clearer. It boils down to this: The researchers found that mutant Cit+ strains arose over the course of time in their long term growth experiment. They wanted to learn something about how that strain acquired this ability. They wondered, 'Was this the result of a single, very low frequency mutation or did some other mutation have to precede it in earlier generations, followed by the final mutation(s) that allowed the cells to grow on citrate?'

:If the Cit+ change required a single mutation with a low but constant probability over time one would expect Cit+ mutations to be distributed across cells taken from any generation of the experiment. On the other hand, if a 'potentiating' mutation had to arise at some point in the cultures before the final Cit+ mutation could function, then one would expect the probability for Cit- cells to mutate to Cit+ cells would increase with samples of cells taken from later cultures.

:Blount ran the experiment and found that Cit+ mutants arose more frequently in cells taken from later generations in the culture. The Monte Carlo resampling tests were used to assess how likely the pattern of results would fit the models. The statistical significance (smaller P-values mean greater significance), was calculated for each experiment and the combination of experiments. The numbers suggest the distribution of mutants was not randomly distributed across the experimental generations, they tended to appear in cells taken from the later generations. This would argue against the single-mutation, constant, low-probability hypothesis. It appears that a pre-adaptive mutation had to have arisen first, followed by the mutation that finally allowed the cells to utilized citrate. Future work in Lenski's lab will focus on trying to identify the various mutations involved. I hope this brief explanation helps.--[[User:Argon|Argon]] 20:25, 15 July 2008 (EDT)

==What I have noticed==
I tell you what strikes me the most about this discussion - Its watching evolutionists do whatever they can to keep their precious beliefs above question or scrutiny. Its as if they know that it will come toppling down so they must resort to side-lining the hard questions.
[[User:JJacob|JJacob]] 21:43, 15 July 2008 (EDT)
==[[Social Text]]==
This well known hoax ''does'' support Aschlalfy's position that the paper was reviewed by PNAC ''rather quickly'', since (apparently) the paper supported the editors/reviewers point of view. Perhaps this would have beeen better placed on the [[Richard Lenski]] page rather than here since this deals only with the 6/10/08 PNAS paper ''per se'' and not the review process. [[User talk:MargeryCampbell|Marge]] 12:45, 16 July 2008 (EDT)

:I fail to see the relevance of the Sokal Hoax here. You should also know that Social Text was not a peer-reviewed journal when Sokal submitted his paper. That doesn't excuse the credulity of its editors, but Sokal's paper was not sent out for review. Not to say that it would have mattered, postmodernism is just meaningless word salad.[[User:Gerlach|Gerlach]] 13:39, 16 July 2008 (EDT)