And the Winner is…

I’ve evaluated the fits submitted for the first Systemic Challenge radial velocity data set, and the winner is Chris Thiessen.

In addition to being dynamically interesting, the configuration proved to be very tough to crack. The challenge 001 system is one where a Keplerian model can reach a low chi-square, which then skyrockets when the planets are actually integrated through their orbits.

I’m travelling today, without full access to the Internet. In the next day or so, once I get back to the office, I’ll put up a more detailed post which looks at what’s really going on in the first challenge system. We’ll also release the second challenge data set (which is equally interesting, but a lot more tractable).

last call

If you’ve been working on the first systemic challenge system, please submit your fit as an ascii file to the e-mail address listed on the web-page given in the Sky and Telescope article. At 00:00 UT Sept. 14th (JD 2453992.5) I’ll close the submissions and see who wins the Star Atlas.

Don’t hesitate to submit if your chi-square is still far from 1.00. Our first challenge system turned out to be a bit harder than we anticipated. Next week’s system will be equally interesting from a dynamical point of view, but will be a lot easier on the ol’ CPU.

HD 208487

Image Source.

An relevant paper showed up on the astro-ph preprint server this morning, “A Bayesian Kepler Periodogram Detects a Second Planet in HD 208487” by P. Ç. Gregory of the University of British Columbia.

Gregory employs a technique known as the parallel tempering Markov Chain Monte Carlo algorithm to argue that the HD 208487 data set contains two planets. The first planet (which was previously announced by Tinney et al. 2005, and confirmed by Butler et al. 2006) has a period of 130 days and a minimum mass 37% that of Jupiter. The second planet in Gregory’s model lies out at a period of 908 days, and has 46% of Jupiter’s mass.

Interestingly, the console does not recover Gregory’s parameters precisely, but it does find a fit that’s extremely similar. (I just uploaded the fit to the systemic back-end.) The radial velocity reflex curve looks like this:


wheras the planetary configuration (at the moment when the first radial velocity data point was obtained on Aug. 8th, 1998) looks like this:

It’s interesting to look at the fits for the latest HD 208487 dataset that have been submitted by participants in the systemic collaboration. At the moment, there are six different fits:

On September 4th, mikevald submitted a 2-planet fit that is a close analog of the one published by Gregory. In the last several days, dstew and andy have also turned in fits that have essentially the same configuration as obtained by Gregory. That’s definitely cool.

In addition to the five fits that look like the Gregory configuration, with the outer planet at a period of P~1000 days, there’s also a completely different take on the system that was submitted this morning by Olweg. In the Olweg fit, the second planet lies interior to the known planet, and has a period of only 29 days. The chi-square is less than one, indicating a slight degree of overfitting. When overfitting occurs, it can easily be remedied by a slight random perturbation of the parameters. It’s very interesting that this fit was completely missed by the Bayesian Kepler Periodogram, so I thought I would have a closer look at Olweg’s model system.

The Olweg radial velocity curve is radically different from the Gregory fit:

The 28.68 day inner planet has a mass of 0.16 Jupiter masses (about 50 Earth masses) and travels on an orbit of modest (e=0.18) eccentricity. There’s a fair amount of planet-planet interaction in this system over the time scale of the radial velocity observations. By the time the fit reaches the end of the data set, there’s a noticeable difference between the keplerian model fit and a self consistent (integrated) model fit:

The system is stable, however, when I did a short test integration of 100,000 years. The secular interaction between the two planets causes the two orbits to execute a complicated dance over a timescale of several thousand years, with the periastron angle of the inner planet orbit mostly librating around an anti-aligned configuration.

As I’ve remarked in an earlier post, we’re currently in the progress of upgrading the downloadable console so that it will be capable of computing estimates of the uncertainties in the orbital elements of a fit. A good way to generate uncertainties in this context is to use the so-called bootstrap method. In the bootstrap, one re-draws the original radial velocity data set with replacement, thus producing alternate realizations of the data in which a fraction of the points appear more than once, and in which a fraction do not appear at all. One then fits to these new datasets, thereby building up distributions for each orbital element. (For more detail, see this paper, which describes in detail how this procedure was applied to the radial velocity data set for HD 209458.) When I run a self-consistent bootstrap analysis based on the Olweg fit, I get the following mean values and standard deviations for the parameters:

This fit is thus quite well constrained, and is a completely viable competing model for describing the hd208487 planetary system. I think the situation here really underscores the value of the systemic collaboration. Many radial velocity data-sets can be fitted by completely different models that offer equally robust fits to the data, while simultaneously maintaining small uncertainties on their bootstrap-estimated parameters.

So how do we know which HD 208487 system (if either) is correct? I’m hoping that the Monte Carlo simulation that will make up phase II of the systemic project will give a great deal of insight into when a particular orbital model can be deemed secure.

desiderata

strands of wheat

Image Source.

It’s good to see that users are still streaming into the systemic collaboration, and activity on the back-end is staying strong. The catalog of submitted radial velocity fits is now approaching 1,000 entries, and nearly every data-set has at least one fit. We need more users, though. Both Stefano and Eugenio have been working very hard behind the scenes to engineer improved usability for the site. There are a lot of items that are still on our plate, but progress is definitely being made.

We can now internally query the database of submitted fits to statistically characterize the planetary models that are being submitted. Once this functionality is fully tested, it’ll be made available to all users on the site. For example, here’s a plot of eccentricity vs. period for all of the fits submitted with 0.8 < chi-square < 2.0: aggregated P-e diagram

It’s interesting to compare this with the plot that one can produce at exoplanet.eu based on the static catalog of published planets:

One immediately notices that the diagram produced from the back-end data is populated in the upper left hand corner, whereas this region in the published catalog is completely cleared out. Note that planets in this region are known from theoretical arguments to be tidally circularized… (tune in soon for more on that issue).

One final note. The downloadable console is now much lighter. The large filesize of the previous version was due to the very extensive synthetic data sets for alpha Centauri. If you want the alpha Centauri velocities, the fully loaded console that contains the alpha Centauri data is here. Within a day or so, we’ll be updating the downloads page to reflect this change and to give more guidance for International Windows users.

New Texas V’s

Image Source.

Data data data. Robert Wittenmyer and his colleagues at the University of Texas have just posted a paper on astro-ph that contains a slew of new radial velocities for several famous planet-bearing stars, including 47 UMa, 14 Her, and 16 Cyg B. The velocities are all tabulated in the paper, so we’ll have them up on the systemic backend very shortly. [I’ll post a comment to this post when they’re up on the site. If you’re totally gung ho to get them right away, you can extract them from the posted latex file at astro-ph, and then add them manually to systemic’s datafiles directory.]

We always try to add new radial velocity data sets as soon as they become publicly available, and lately, these updates have been occurring roughly once per week. For the time being, the simplest way to get your fresh V’s is to rename your old systemic directory, and then download a new console. When the new console and catalog data are downloaded and unzipped, you can copy any previous fits and soundfiles that you’ve created into the new fits and soundClips directories.

The data in the Wittenmyer paper come from both the Harlan J. Smith 2.7-meter telescope and the Hobbey-Eberly 9.2-meter telescope. The cadence of the Smith telescope observations typifies the usual pattern of radial velocity survey data. The individual points are spaced essentially randomly in time, with many days separating each point. The Hobbey-Ebery data, on the other hand, are quite different. These data are much more densely sampled, and many nights contain several velocities in succession. In many stretches, the star is observed every few nights. This pattern results from queue-scheduling, which enables very intensive monitoring of systems that are of particular interest. I think queue scheduling is the wave of the future, and in the systemic simulation, we’ll have many synthetic data sets whose cadences correspond to the queue-scheduled approach.

The most prominent planet orbiting 14 Her has been known since the late 1990s. This world, known as 14 Her “b”, has a minimum mass about 4.6 times that of Jupiter, and a period of ~1770 days. If it were in our solar system, it would orbit in the asteroid belt. The parent star 14 Her is about 90% as massive as the Sun, and is more than twice as metal-rich. Given the planet-metallicity connection, it’s absolutely no surprise that 14 Her has a heavy-duty planetary system. I bet that 14 Her “b” has a very interesting system of satellites.

It’s pretty clear from the one planet fit that 14 Her “b” is not the only planet in the system, and over the weekend, several systemic users have submitted interesting fits to the data that reduce the chi-square by adding a second planet. For example, on August 30, user mikevald uploaded a two-planet fit in which the second planet, 14 Her “c”, has a period of 6159 days and an eccentricity e=0.52. This model currently fields the lowest chi-square statistic of any of the submitted 14 Her fits. The orbits in this best-fit system are crossing, however, indicating that the model may not be dynamically stable over the long run. On September 5th, allanfloering submitted a fit with nearly as good a chi-square, in which the outer planet has a 14,669 day period and an eccentricity e=0.09. Allanfloering’s world, if it exists, lies 11.77 AU from 14 Her, out at a Saturn-like distance.

Wittenmyer et al. show that the addition of their new 14 Her data suggests that 14 Her “c” has a period of order 6900 days, albeit with a low eccentricity. In their models, “c” and “b” may be participating in 4:1 resonance. A quick fit on the console with the Wittenmyer et al data included gives a radial velocity curve that looks like this:

corresponding to a planetary configuration that has an outer planet with a modest eccentricity e=0.20.

As soon as the data go up on the site, feel free to try working up improvements. It will be interesting to see how many fits to the full 3-telescope data set are participating in 4:1 resonance.

Web 2.0

fenceposts at ucsc

Hey ya’ll, there’s a whole lotta fittin’ goin on out there in the back 40.

Seriously, though. We’re really seeing a great response from users who are contributing their efforts. Nearly 200 people have registered on the back-end during the past few days, and over 750 different radial velocity fits have been uploaded. Hopefully we’ll see that work continue to flow in, and everyone has been showing admirable patience as we smooth out the inevitable rough spots which began to show up as soon as we had a surge of real users on the site.

If you’re arriving by way of the Sky and Telescope article, you’ll notice that the full universe of 100,000 synthetic stars is not yet listed on the systemic backend. During September, we’re still carrying out the first phase of our planned research effort, which consists of accumulating a wide variety of fits to the full collection of actual, published radial velocity data sets. Very soon, we will have accumulated enough fits to be able to present a dynamic, interactive catalog of candidate planets. A query-based dynamically generated planetary catalog will allow a variety of very interesting questions to be answered. For instance, by how much can one deflate the famous eccentricity-period diagram, while still demanding a prespecified goodness-of-fit for all of the candidate planets?

generated at exoplanet.eu

At the moment, such questions are hard to answer, because (other than here at oklo) there is no consolidated repository of radial velocity data and associated self-consistent fits.

In order to make dynamically generated planet catalogs scientifically useful, we’re going to have to provide several more tools to the users. As I mentioned yesterday, the console will soon be multi-threaded, which will make it easier to use for high-performance work. In the interim, however, you can have the console print a stream of diagnostic messages by launching it from the command line. For example, on linux or OSX architectures, open a terminal (shell), cd to the systemic directory, and type java -jar systemic.jar at the prompt . The diagnostics provide a running update of the progress of the console as it produces fits to the data set.

We’ll also soon be providing a long-term integration window that will allow users to verify that their model systems are dynamically stable. It’s alarmingly easy to find multiple-planet fits to radial velocity data sets that have low values for the reduced chi-square statistic, but in which the planets experience dynamical disasters (collisions, ejections, close encounters, etc.) on a time scale that is short in comparison to the known age of the parent star. Indeed, most of the candidate stars in the back-end catalog are more than 2 billion years old. Young stars tend to be rapidly rotating, which broadens their absorption lines and makes radial velocity measurements less accurate. Rapidly rotating stars also tend to have elevated levels of magnetically driven chromospheric activity, which adds additional noise to the velocity estimates.

And finally, the console needs to provide error estimates on the orbital parameters that it generates. This is best done using the so-called bootstrap method, which we’ll discuss in an upcoming post.

Systemic Challenge — data set #1

Image Source.

We’re continuing to see a strong influx of new users and activity on the systemic backend. Thanks to everyone who’s taking part! If you’re a first-time visitor to the Systemic Project website, please read the weblog entries that follow this post. They contain the information you need to start participating, and they give a recent day-by-day overview of the project developments.

Now that we’ve got an active user-base for the systemic console, we’re pleased to release the first Systemic Challenge synthetic radial velocity data set. This data set corresponds to a realistic simulated planetary system that is both scientifically interesting and non-trivial to fit.

Sky and Telescope is sponsoring the world’s first radial velocity fitting contest in connection with our challenge system. The person who submits the self-consistent (integrated) fit to the data having a chi-square value closest to one will receive a paperback edition of the Millennium Star Atlas (a $149.95 value).

Continue reading

consolidation

Image Source.

Wow! The American Scientist and Sky and Telescope articles are clearly getting the word out. We’ve been seeing a significant increase in traffic on the oklo.org site, both in terms of visits (yellow bars) and bandwidth and page views (green and blue bars). The bandwidth increase is especially gratifying. It reflects the fact that many users are registering on the back-end, downloading the console, and submitting fits. As I write this, new and interesting fits for a variety of different radial velocity data sets are rolling in to the star catalog. Our goal of fostering original, public-participation exoplanet research is starting to be realized, and I want to thank everyone who’s lending a hand.

Late August stats.

If you’re a first-time visitor to the Systemic Project website, please read the blog entries that were posted prior to this entry. They contain the information you need to start participating, and they give an overview of the current project status. If you are a return visitor, please have a look at the updated back-end. Stefano has made a number of code and design improvements that streamline the workflow and make the site easier to navigate.

On to some planet issues. The Mu Ara (HD 169061) system, which contains four known planets, is shaping up to have significant implications for the systemic project. Intense interest in the system has been spurred by a recent paper from the Swiss group (Pepe et al. 2006) that presents a self-consistent 4-planet model. Pepe et al.’s orbital fit (given in their Table 1) provides an excellent match to the radial velocity data sets, but when they carried out a long-term integration of the system, they found that the gravitational interactions between the planets lead to catastrophe after 76 million years. The parent star Mu Arae has an estimated age of 6.4 billion years, so clearly we don’t yet have a full understanding of what’s going on with this system.

The discord within the Pepe et al. model is provided by the two middle planets, one of which has a 310 day orbit, and the other which orbits in 643 days. The planets are on the edge of the 2:1 mean motion resonance, with the practical consequence that they experience a strongly chaotic orbital evolution. The orbits change eccentricity and orientation on a timescale of only decades:

I’ve made a movie that tracks the evolution of the orbits over 528 years. Here are links to a .mov version (288 kB) and an .mp4 version (1.5 MB). It’s clear from the movie that the interaction is both complicated and unpredictable. The planets display no catastrophic excursions on the 500 year timescale of the movie, but eventually, they experience orbit crossings leading to a likely ejection of the inner 0.5 Jupiter mass planet.

The Mu Ara dataset HD169061_B06P06CH on the console back-end combines both the Pepe et al. data as well as the most recent data from Butler et al. 2006. I’m hoping that someone can get a stable, self-consistent, low chi-square fit to this combined data set. Such a fit would give the best available view of what’s going on with the system, and would underscore the scientific relevance of the systemic project.

more updates

We’ve been seeing a nice increase on traffic here at oklo.org as the Sky and Telescope and American Scientist articles show up in mailboxes and on newstands. If you’re new to the site, welcome aboard, and please read the last several posts. They give a brief overview of the Systemic Project, and tell you what you need to start fitting systems.

As you may have noticed, we’re hard at work improving the usability of the site. Stefano, in particular, has done an amazing amount of work on the back-end over the past several days. It really is taking shape as a genuine research environment, and we absolutely are urging you to try it out while we’re putting the definitive users manual together. You can’t break anything, and if you post questions, we’ll make sure they get answered on a timely basis.

A few news items:

(1) There are two separate channels for registration on systemic. The first, accessed through the “login” tab on the site header above, is part of the WordPress package that runs the oklo blog. Registration on the blog allows you to comment on the oklo.org posts. The second, accessed through the “backend” tab on the site header, gives you access to the collaborative php-based environment that constitutes the systemic backend. You can register for either or both, and you don’t need to give your real name or any real-world identifying information other than an e-mail address.

(2) We’ve been working to design the first systemic challenge radial velocity data set, which will be released on Monday September 4th. The user who finds the fit with a reduced chi-square closest to unity will win a $149 Sky Atlas from Sky and Telescope. Both professional and amateur planet hunters are encouraged to participate, but given the groundswell of activity that we’re seeing on the systemic back-end, and given the console’s ability to carry out self-consistent fits, the smart money is on an amateur winner.

Updated back end

Image Source.

Stefano (while in the midst of writing his astrophysics thesis in Bologna!) has somehow found the time to implement a whole slew of very cool improvements to the Systemic back end. New aspects include a search function, a personal “fits” library, and a variety of interactive features designed to aid collaboration and make the user experience more rewarding.

We’re working on a full user manual for both the downloadable console and the back end, but in the meantime, we’re really urging users to (1) download the console, (2) create a free account on the back end, (3) work through the systemic console tutorials one, two, and three, (4) upload fits, and (5) start collaborating. There are a number of real systems in the star catalog which can be profitably characterized and improved. I’m also very interested to see what people come up with for the datasets systemic001 and systemic002.

Do give it a try, and please give us feedback, either in the comments section on the back-end, or in the comments section for this post.

Two tips: (1) For Mac-users, Firefox provides a better interface to the back-end than does Safari. (2) At the moment, the user-interface looks better if you resize to a wider browser window.