Comments on: What Counts as a Big Effect? (I) http://gothamschools.org/2009/04/24/what-counts-as-a-big-effect-i/ Fri, 10 Feb 2012 01:30:00 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: “All Teachers Are Not Created Equal,” Should Teacher Unions Explore the Impact of “Value-Added Modeling” Alternatives to Traditional Salary Schedules? « Ed In The Apple http://gothamschools.org/2009/04/24/what-counts-as-a-big-effect-i/comment-page-1/#comment-114480 “All Teachers Are Not Created Equal,” Should Teacher Unions Explore the Impact of “Value-Added Modeling” Alternatives to Traditional Salary Schedules? « Ed In The Apple Fri, 08 May 2009 02:38:46 +0000 http://gothamschools.org/?p=13131#comment-114480 [...] are “significant.” Aaron Pallas, at Gotham Schools is instructing us, a sort of Statistics 101.   As the evidence mounts we can conclude, not surprisingly, that “all teachers are not [...] [...] are “significant.” Aaron Pallas, at Gotham Schools is instructing us, a sort of Statistics 101.   As the evidence mounts we can conclude, not surprisingly, that “all teachers are not [...]

]]> By: eduwonkette http://gothamschools.org/2009/04/24/what-counts-as-a-big-effect-i/comment-page-1/#comment-105830 eduwonkette Sat, 25 Apr 2009 14:04:07 +0000 http://gothamschools.org/?p=13131#comment-105830 Fabulous figures. Thank you!!! Fabulous figures. Thank you!!!

]]> By: ceolaf http://gothamschools.org/2009/04/24/what-counts-as-a-big-effect-i/comment-page-1/#comment-105258 ceolaf Fri, 24 Apr 2009 15:00:47 +0000 http://gothamschools.org/?p=13131#comment-105258 Aaron, This is a great explanation, and I'd like to add a couple of things. 1) To get a sense of scale, the test that we all know best is probably the SATs, whose standard deviation is supposed to be 100 points in each test (i.e. verbal or math). That means that this latest Hoxby study show effect sizes that are proportional to average of 9 more points on the SAT math test (i.e. 0.09 * 100) and 4 more points on the SAT verbal test (i.e. 0.04 * 100). 2) Is that a lot? I'll leave that for you get into a bit more next next week. As you said, however, it *is* statistically significant, which is not a comment on effect size. I want to say that again: statistical significance is *not* a comment on effect size. So, a lot of studies show "small but [statistically] significant" effect sizes. 3) I very much like that you pointed out that a given difference as measured in standard deviations means different things in term of percentiles at different points along the distribution. However, studies like this recent one from Hoxy about differences from the mean or between means of different populations. She is writing about the average scores of all NYC charter students compared to the average scores of all NYC non-charter public students -- or should be. These kinds of studies are looking near the mean (i.e. the highest point of the bell curve). Aaron,

This is a great explanation, and I’d like to add a couple of things.

1) To get a sense of scale, the test that we all know best is probably the SATs, whose standard deviation is supposed to be 100 points in each test (i.e. verbal or math). That means that this latest Hoxby study show effect sizes that are proportional to average of 9 more points on the SAT math test (i.e. 0.09 * 100) and 4 more points on the SAT verbal test (i.e. 0.04 * 100).

2) Is that a lot? I’ll leave that for you get into a bit more next next week. As you said, however, it *is* statistically significant, which is not a comment on effect size. I want to say that again: statistical significance is *not* a comment on effect size. So, a lot of studies show “small but [statistically] significant” effect sizes.

3) I very much like that you pointed out that a given difference as measured in standard deviations means different things in term of percentiles at different points along the distribution. However, studies like this recent one from Hoxy about differences from the mean or between means of different populations. She is writing about the average scores of all NYC charter students compared to the average scores of all NYC non-charter public students — or should be. These kinds of studies are looking near the mean (i.e. the highest point of the bell curve).

]]>