Comments on: still feel gone http://oklo.org/2006/07/21/still-feel-gone/ characterizing planetary systems Fri, 18 Nov 2011 22:27:52 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: darin http://oklo.org/2006/07/21/still-feel-gone/comment-page-1/#comment-232 darin Sun, 23 Jul 2006 01:07:53 +0000 http://oklo.org/?p=112#comment-232 I completely agree with your conclusion that 51 Peg c is almost certainly a spurious result. However, the idea brings up a concept that I hope you would discuss in future posts. There are statistical probabilities that can be associated with fitting (radial velocity) data. Adding another planet (equivalent to adding 7 parameters to the fit) should significantly reduce the chi-square. ** The goal of RV fitting (with the console or otherwise) is to find the smallest value of (reduced) chi-square possible for a particular data-set. ** If there exists a fit with a chi-squared of 2 or 3, then solutions that have chi-squared of 4 or 5 remain in the realm of possibility: future data may justify these fits. However, a solution that has a chi-squared of 20 or 50 is extremely less likely to be correct that a solution with chi-squared or 2 or 3. Furthermore, adding another planet can be statistically justified. The Monte Carlo method described in post 61 is one excellent way. Another way is to use an "F-test", where two different solutions with different chi-squares and different numbers of planets can be compared. To get a statistically significant improvement, an additional planet is only justified if the chi-square is reduced by quite a bit. Even a reduction of the reduced chi-square by 5 may not be enough to justify the addition of all those new parameters we're fitting with the console. This is because adding parameters will always have the tendency of improving the fit. Theoretically, any RV curve could be virtually perfectly modeled by ~100 planets of all the right size, phase, and orientation, but these are not justified if a 1-planet fit has a reduced chi-squared near 1 (especially given the natural error induced by stellar jitter). Perhaps I have gone on too long. Maybe all the blog readers know more about chi-squared than I do, but those who use the console should know their ultimate goal: find a fit to RV data with the fewest planets and the lowest chi-square possible. A fit that is lower than the chi-square reported in the planet discovery paper is interesting, especially if the planetary parameters are different than those proposed, but often not if an additional planet is required. I completely agree with your conclusion that 51 Peg c is almost certainly a spurious result. However, the idea brings up a concept that I hope you would discuss in future posts.

There are statistical probabilities that can be associated with fitting (radial velocity) data. Adding another planet (equivalent to adding 7 parameters to the fit) should significantly reduce the chi-square. ** The goal of RV fitting (with the console or otherwise) is to find the smallest value of (reduced) chi-square possible for a particular data-set. ** If there exists a fit with a chi-squared of 2 or 3, then solutions that have chi-squared of 4 or 5 remain in the realm of possibility: future data may justify these fits. However, a solution that has a chi-squared of 20 or 50 is extremely less likely to be correct that a solution with chi-squared or 2 or 3.

Furthermore, adding another planet can be statistically justified. The Monte Carlo method described in post 61 is one excellent way. Another way is to use an “F-test”, where two different solutions with different chi-squares and different numbers of planets can be compared. To get a statistically significant improvement, an additional planet is only justified if the chi-square is reduced by quite a bit. Even a reduction of the reduced chi-square by 5 may not be enough to justify the addition of all those new parameters we’re fitting with the console. This is because adding parameters will always have the tendency of improving the fit. Theoretically, any RV curve could be virtually perfectly modeled by ~100 planets of all the right size, phase, and orientation, but these are not justified if a 1-planet fit has a reduced chi-squared near 1 (especially given the natural error induced by stellar jitter).

Perhaps I have gone on too long. Maybe all the blog readers know more about chi-squared than I do, but those who use the console should know their ultimate goal: find a fit to RV data with the fewest planets and the lowest chi-square possible. A fit that is lower than the chi-square reported in the planet discovery paper is interesting, especially if the planetary parameters are different than those proposed, but often not if an additional planet is required.

]]> By: andy http://oklo.org/2006/07/21/still-feel-gone/comment-page-1/#comment-231 andy Fri, 21 Jul 2006 08:50:16 +0000 http://oklo.org/?p=112#comment-231 Talking of modifying the data sets, I (fairly arbitrarily) threw out the first 14 data points in 55cancri_1.vels. Interesting what happens to the residuals: 55 Cancri e disappears and is replaced by a 390-day planet. Talking of modifying the data sets, I (fairly arbitrarily) threw out the first 14 data points in 55cancri_1.vels. Interesting what happens to the residuals: 55 Cancri e disappears and is replaced by a 390-day planet.

]]>