Image Source.
The fourth systemic challenge turned out to be somewhat less challenging than the first three. Quite a few entrants figured out that the data-set corresponds to our own solar system. Among a large number of excellent models, Mark Kilner turned in the fit with the lowest chi-square: 1.0401. In addition to Jupiter, Saturn, Earth, and Venus, he topped off his system with a spurious Mercury-mass planet in a 5.62 day orbit, which allowed him to take the prize. Nice one, Mark!
Eugenio created the challenge 004 synthetic data set after a conversation in which we decided that it’ll soon be feasible to push the precision of the radial velocity method down to an instrumental error of 0.1 m/s. Even more optimistically, we assumed that the Sun, viewed from afar, exhibits negligible radial velocity noise (more on that soon).
Our Solar System, expressed in the Jacobi orbital elements used by the console, is given by:
The true three-dimensional model that Eugenio actually integrated to produce the synthetic data set also includes the correct values for the planetary inclinations and nodes. Because of the sin(i) degeneracy for Keplerian orbits, the current version of the downloadable systemic console does not include the inclinations and nodes as fitting parameters.
The synthetic data set was created with the KeckTAC program, which mimics realistic observing strategies. In an all-out effort on a particular star, one would combine repeated individual observations to get a composite observation that averages over the effect of short-period oscillations (p-modes) of the star itself. This is the strategy that is being currently used by the Swiss team in their campaigns on stars such as HD 69830 and HD 160691. In the challenge004 dataset, there are 1171 radial velocity measurements spread out over 24 years.
Eugenio describes the procedure he used to fit the data:
The periodogram (and the data) shows Jupiter clearly. Saturn appears as a trend, but the periodogram of the residuals after fitting Jupiter gives a good guess for Saturn’s period. After removing Saturn, Earth pops out in the residuals periodogram. I did not find it easy to fit Jupiter, Saturn, and Earth, but after succeeding, Venus very clearly appears in the residuals. I kept on fooling around with the 4-planet fit to see if there was any chance of finding Mars even though the RMS was telling me that 4 planets was the best that I would likely do. I was hoping that N would be large enough to let me get Mars, but I was not able to see a (significant) signal in the residuals periodogram. If anything, Mercury seemed to be more easily detectable. However, after fooling around with the eccentricities of Saturn, Earth, and Venus, the (weak) signal for Mercury disappeared.
Image Source.
With the contests wrapped up, we’re now in the business of getting the fits completed for the Systemic Jr. data set. Eugenio recently incorporated an F-test module into the console, which can be used to determine whether the addition of a planet is warranted. We’ll have a post up shortly that explains in detail how this works. In the meantime, see the discussion on the backend, or download a new console and give its new modules a whirl.
Thanks, Greg. What a pleasant surprise for a Monday!
I tried several different fits which also included Mars in the mix, but the integration process kept pushing it into an Earth-crossing orbit (good for the ChiSq, but not so good for the inhabitants of synthetic Earth!).
Hi Everyone,
This system shows some of the obsticles that radial velocity observers have to surmount. Users of the console have already experienced most or all of these difficulties. One is to observe for sufficiently long to detect long-period planets. Another is that noise makes it difficult to detect low amplitude planets unless there is enough data (and/or the noise level is small). Yet another one is that it is sometimes very important, but not always easy, to get a good fit for 1, 2, 3, … planets in order to reliably detect an additional one. I also alluded to one rule of thumb in my comments above. Although adding an additional planet may decrease chisq, if the RMS of the fit without the additional planet is already close to the median value of the uncertainties in the data, the fit with the additional planet should typically have little significance. There will probably be more on this when Greg posts on the F-Test module. In the mean time you can look at the help section in the new module for additional information.
Eugenio
Hi Everyone,
The bottom line for my last comment (most console users should already know this) is that it’s not easy to find a Solar System analogue.
Eugenio