Search This Blog

Wednesday, July 30, 2008

Peer review

Peer review has been mentioned a lot within the discussions surrounding the STFC funding crisis. Notably it was mentioned during the House of Commons debate a few weeks ago, though much of the discussion came across to me as somewhat fuzzy in its logic.

In the debate there was much talk of whether peer review is the best way to go, how no system is ever perfect and how dangerous it could be for the select committee or anyone else to attack peer review.

I think this missed the point completely; the underlying assumption throughout the debate was that peer review describes a single process, whereas we all know that peer review is something of an umbrella.

This bothered me before when there were admonishments from some that the community should be careful when attacking peer review. It is a false answer; who in the community has really attacked the concept of peer review? Anyone? I have heard plenty of people attack how STFC implemented a process that they claimed was peer review. You see this is just more spin to deflect a real question and it bothers me that we have not heard more about this. So what is the real problem with 'peer review'?

In my view the issue with peer review in the STFC was not that 'peer review' as a concept was fatally flawed, rather that the practical implementation of peer review was rubbish in the eyes of many of us.

With the best will in the world a small panel with no advisory structure could not be considered to be effective peer review for the wide-ranging programme that STFC was responsible for. PPAN did the best they could but let's face it they are only human.

Arguments that these are intelligent people and so they were perfectly capable don't hold water; yes they are intelligent but that is not enough in a situation where community confidence is essential and where people's livlihoods may be impacted. You need intelligence and breadth of experience and familiarity; a proper balance.

When someone argues that smaller committees are better because they are efficient and don't try to please everyone you know they have not really thought through the implications of what they are saying with regard to a smaller panel. Yes, a smaller panel might reach a decision quickly but one lacks the confidence that they might reach the correct decision.

I note that peer review came up again in the letter from Phil Willis MP to John Denham:

It is not clear why the Government goes on to discuss peer review in this section, since peer review was not mentioned in recommendation 6 or its preceding text and was discussed later in the report. Be that as it may, the Government’s assertion that we criticised “the outcome of STFC’s peer review process” and “those researchers who have undertaken it” is an inaccurate paraphrasing of the serious concerns we raised in relation to STFC’s peer review system and decisions made by STFC. We did not criticise the outcomes of STFC’s peer review. Specifically:
(a) on the International Linear Collider, we did not comment on the scientific justification for withdrawal, but raised some concerns that had been put to us during the inquiry;
(b) on Gemini, we did not consider the merits of STFC’s decision, but the way it went about making its decision, or as it turned out, indecision, public;
(c) on solar-terrestrial physics, we questioned Professor Mason’s explanation for the withdrawal of funding, and suggested that STFC renege on that decision until its community had been properly consulted.
Neither did we criticise the members of the peer review panels. On the contrary, we acknowledged that STFC’s peer review committees “have a difficult job to do” and that “we do not doubt the integrity of the individuals who make up those Committees” (p 32 of our report).
Sadly the Secretary of State's reply is less than promising and seems to boil down to a game of statement of 'I'm not going to debate this with you, game over'.

I suppose he has been too busy running around letting everyone know that Gordon Brown is still great and we should stop attacking him because the conservatives are much worse.

The letter is worth reading in full as is the one to Prof. Keith Mason, who to his credit (though not much) gave a more fulsome response to the points raised.

Recognising service

The Royal Astronomical Society has called for nominations for its 2009 awards. Today is the last chance to send in a nomination.

The award that stuck out to me was the RAS service Award. This is intended to:

honour individuals who, through outstanding or exceptional work, have promoted, facilitated or encouraged the sciences of astronomy or geophysics and developed their role in the life of the nation but whose achievement does not fall within the criteria of the Society's other awards.

I was thinking of nominating both Prof. Keith Mason and Prof. Richard Wade as I don't think anyone else has done quite so much in terms of uniting the astronomy and space science community.

Thursday, July 24, 2008

Substorms solved - or are they?

The THEMIS team has announced that the primary mystery of substorms has been resolved:

"We have discovered what makes the Northern Lights dance," declares UCLA physicist Vassilis Angelopoulos, principal investigator of the THEMIS mission. The findings appear online in the July 24 issue of Science Express and in print August 14 in the journal Science.

What is a substorm?

Well, broadly speaking they are massive bursts of energy from the night-side part of the Earth's magnetic field; here the field is stretched back into space, almost like a comet tail. Energy from the solar wind gets carried into this magnetotail in the form of increased magnetic field strength and energetic charged particles until at some point the energy is released, a good portion of it carried Earthward.

Some of those particles dive along the Earth's magnetic field and produce the bright and dynamic auroral displays (e.g. see the pretty picture) that can be seen off to the north. Some of them hang around a big longer in the inner magnetosphere and drift around to the dayside. Some even get accelerated and add to the population of relativistic electrons in the radiation belts; these are the really dangerous ones.

The primary question about substorms (though only one of many) is: 'how is the energy release initiated?'

There are two principle competing theories* and the big difference between them is the order in which physical processes occur in the magnetotail. It has been known for a long time that magnetic reconnection** plays a significant role (sorry to my astronomy colleagues as I have been reliably informed that you get bored hearing about this). What was not clear was whether this initiated the substorm process (or more accurately the substorm expansion phase) or whether it occurred after some other process had already started the ball rolling.

Over the years the different physics that occurs within the substorm have pretty much all been observed (well, as far as we know...), but knowing in which order they occur has always been tricky. To find that out we would need probes along the magnetotail recording simultaneously as the substorm occured, plus a whole network of ground based observatories so that we can interpret the satellite data in proper context.***

That's what THEMIS is: five satellites spread out along the tail with a huge and impressive network of ground instrumentation across North America (and a couple in Europe for good measure - one of them operated by a UK scientist). Now, it seems, this ground and space based mission has come good.

The discovery came on what began as a quiet day, Feb 26, 2008. Arctic skies were dark and Earth's magnetic field was still. High above the planet, the five THEMIS satellites had just arranged themselves in a line down the middle of Earth’s magnetotail—a million kilometer long tail of magnetism pulled into space by the action of the solar wind.

That's when the explosion occurred. A little more than midway up the THEMIS line, magnetic fields erupted, "releasing about 1015 Joules of energy," says Angelopoulos. "For comparison, that's about as much energy as a magnitude 5 earthquake."
And the satellites and ground-based cameras saw it from start to finish:

"For the first time, THEMIS has shown us the whole process in action—from magnetic reconnection to aurora borealis," says Sibeck. "We are finally solving the puzzle of substorms."

So it looks as if we can tick a box: magnetic reconnection is the first step in the process. Or at least that is my interpretation of this press release. This could be the answer to a question that has been around for over 30 years.

However, I would bet that this is far from the definitive answer.

I saw talks in the special Substorms in the Age of THEMIS session that we held at NAM that suggested that the story is actually a whole lot more complicated. One in particular hinted very strongly that two different mechanisms occurred in two substorms that quickly followed one another. I'm keen to see if THEMIS also sees another substorm; one that doesn't follow the same pattern as the February 2008 example. There is time yet.

Plus the substorm debate is far from over; there are many other unanswered questions surrounding the substorm: What triggers the reconnection and why does it happen when it does? What determines the location of the aurora? Why do periodic substorms have such large particle injection regions? etc.

For a look at a some recent work that addresses a different aspect of the substorm issue grab a copy of August's Astronomy and Geophysics, where one of the Rishbeth prize winners was looking at many substorms at Earth. The other prize winner was looking at a product of auroral substorms -kilometric radiation- but on Saturn, rather than Earth. This could be a very useful tool for identifying extra-solar planets that have a magnetic field, something that might well be a prerequisite for complex life.

*A primer on potential candidates

**Reconnection is the process by which two different magnetic fields (pointing in opposite direction) merge to produce a new configuration. In the Earth's magnetotail this means that the field close to the equatorial plane gets pinched together causing a change in shape that propels some of the charged particles Earthward. Essentially the field goes from being stretched out in a long tail, to looking more like a dipole field. This animation demonstrates the basic concept nicely.

***I hereby resist the urge to point out to STFC the stupidity of withdrawing from ground-based STP facilities. Except for this footnote...

Wednesday, July 16, 2008

What was that about STFC and Haldane?

In an earlier post I mentioned that one of the most interesting things to come out of the Estimates Day debate on the Science Budget in the House of Commons was how they referred to STFC in the context of the Haldane principle.

Let me explain.

A question I have heard since this shortfall debacle began is over whether STFC is effectively just a branch of the government, representatives of the science community, or some mixture of both.

If the answer is that they are a branch of the government then it is assumed that they are responsible for exercising government policy, if not then they are beholden, somewhat, to the wishes of the community. Now of course even if they are not part of government they would be foolish to ignore government when they outline science priorities in the context of a spending review.

I think that discussion of the Haldane principle during that debate answered the question.

Since the government believes it is up to STFC to determine what it should spend its money on once the money is handed over it is quite clear that the government takes the view that STFC is a representative of the science community. Any suggestion that they are a de facto branch of government and responsible for executing government policy (outside of earmarked funds) is wide of the mark.

Now again there is the caveat that it would be an unwise research council that ignored government science policy in deciding how to spend its money; however ultimately the views of the scientists are pre-eminent in formulating how the money should be spent.

Of course one then gets into an argument over whether the research councils are truly representative or are we in the territory of a select few deciding what science to do. And who is to say which is the right way to go about its?

But that is another issue completely.

The ongoing struggle

I've been a bit quiet lately but i thought it was about time I discussed a few things STFC related.

I want to point to a couple of very interesting pieces that have appeared in the media recently.

First is a letter to the Times from Prof. Andy Fabian the new president of the RAS. This follows the more positive stories that were appearing in the media concerning the £80m shortfall and the resolution of the programmatic review that seemed to save Jodrell bank.

Whether the folks at Jodrell liked it or not they had become a focal point for this struggle, a fact that highlights the high regard that excellent institution has in UK society.*
Andy highlights that although there is more money to keep Jodrell afloat (though the University of Manchester still has to find a lot of the cash to keep it running) there are plenty of things that are screwed.

Sir, Jodrell Bank undoubtedly now has a more promising future than seemed possible a few months ago (report, July 9) but the crisis for science funding is not over.

Andy also flagged the coming storm; not that the shortfall has not already caused a reduction in grant awarding:

At the same time a £33 million cut to university research groups’ grant funding will mean a steep cut in the number of postdoctoral and postgraduate researchers, so the facilities we still have will not be exploited in the way they should be.
Overall, an excellent letter.

More recently Prof. Steve Schwarz penned an excellent article for Research Fortnight called A lingering sense of struggle in the slog to learn physics lessons.

As chairman of one of the 10 specialist advisory panels that the STFC hurriedly set up in March after its own efforts at ranking projects had been roundly condemned by researchers, I left the meeting still stunned by the events that have unfolded since the council revealed an £80 million hole in its budget last November. And I remain fearful for the future of the UK’s physics and astronomy research.

Once more this looks to the future and wonders whether as well as identifying the lessons to be learnt from the recent balls up, STFC will actually learn thos lessons. Steve is not sure:

But has the STFC really learned these lessons, or does it merely now know what they are? The omens are not good.

Steve wonders about the upcoming strategy and how much input the community will have, and asks sensible questions about the way the new advisory panels will be set up:

Specialist advisory panels are being re-established; in fact, we have heard this for a while now, but implementation does indeed seem to be getting closer. We are told that these panels will be subject-based rather than facility-driven. This is a welcome sign—and a lesson learned. But will there be enough panels? Enough members? Enough influence to be seen as an effective route through which the community can channel its talents and aspirations?

Wise words about the need for adequate expertise in tensioning projects and facilities in the new STFC. The idea for a new model council with as few a people as possible to expediate decision making looks to have been shown to be a dud. You may get decisions made efficiently and quickly but it hardly inspires confidence that the right difficult choices are made.

Effective decision-making needs more than a small set of advisers to cover the enormous breadth of the programme, from running large facilities costing hundreds of millions of pounds at one end, through major international collaborations on fundamental science, to individual research grants at the other.
Prof. Schwarz also takes aim at the much lauded consultation process that followed the rankings in the programmatic review. I have mentioned some issues with this before but Steve was a chair of one of the panels and so has greater insight than most:

In fact, the process was much less effective than it could have been because the community was provided with scant feedback to indicate why projects were ranked where they were. The panels themselves also found information difficult to obtain.

Not surprisingly, community responses often merely re-iterated project objectives or successes. The few responses from projects initially ranked high generally praised the wisdom of that ranking; while the flood of responses from projects ranked low often vented obvious frustration and anger.

One gets the sense that there is less satisfaction with the process than has been aired recently. Let's face it, it did not help that the consultation came after the rankings. Better to have had an initial consultation - or at least solict advice from the wider community, followed by a consultation after the ranking. We have heard that prior discussion has been mooted for next time by STFC.

Steve's article concludes with something for us all to think about in this time of post-programmatic review, as things seem to be calming down:

This is not to smear the integrity of the small team of undoubtedly stressed-out advisers within the present structure. It is to condemn a top-down management style that is trying to operate a ‘one-size-fits-all’ approach for the funding of both science and facilities within a financial straitjacket.

Will the promised Specialist Advisory Panels make a difference? Will they be given the influence to enable the scientific community to work with the STFC? As usual, we can only wait to be told.

It is essential that we have our say on how the advisory panels are set up and not least who goes on them. We must work to ensure that there is adequate representation enshrined within their formation and not just lip-service to the notion. To their credit STFC executives have said that they are only a phonecall or an email away.

*Amusingly I am Cheshire born and bred and have tremendous affection for my home county yet I have never visited Jodrell bank. My year missed out on that school trip.

Friday, July 11, 2008

The nuclear option

Since Andy is now on sabbatical in the States he was not able to attend the town meeting on Tuesday. I was not able to attend either - mostly because I have work to do and it costs a bit to high-tail it to London for meetings; what remains of the funding that supports me can be better spent!

Anyway Andy points to a new furore that is bubbling away as the Nuclear scientists start to get vocal in the media over their loss of funding:

Hang on a tick though, the Telegraph seems to have a story about STFC destroying the country’s nuclear power capability !!! Naturally, STFC have put out a statement explaining that they are reponsible for the nation’s power stations.

Having read the STFC statement I think they have scored quite an own goal here. I can understand why they would want to nip more negative stories in the bud; just as they were probably hoping that this would all start to go away (especially given the positive press regarding Jodrell Bank that has appeared in the press). However they have gone the wrong way about it.

The branch of nuclear physics that STFC supports is the investigation of the internal structure of the nucleus and the nuclear processes that take place in stars (nuclear astrophysics) and does not concern nuclear power or nuclear engineering, or the building or decommissioning of power stations.
At a time when community morale is at an all time low the statement does little to bolster the nuclear community spirit. It won't win awards for attracting students and researchers who might be interested in nuclear physics.

More importantly it undermines any arguments for relevence to society and UK PLC. This is incredibly unwise in a period when we are told that we must demonstrate our worth to justify the funding that we receive.

This could have been handled much better.

Quote of the day

The debate in the House of Commons on Monday night held some little gems but this was the best quote by far:

I was concerned that the STFC may have misinterpreted one of our recommendations, which was to improve communications. I do not think that that meant that it should improve its spin.

The discussion on the Haldane principle and how it applies was actualluy quite fascinating, not for whether Haldane was breached but rather for the context of STFC within that discussion. Very interesting to hear the government and political view.

Thursday, July 10, 2008

Beatles Day

Liverpool is celebrating its most famous sons with its first Beatles Day.

Look, at the risk of going all 'Boris Johnson', this is one thing that really bothers me*. Let's face it, every day in Liverpool is Beatles day. It is hard to escape it as it seems to be the default setting.

Liverpool has a long and interesting history, some of it not great, but there is a lot to be proud of in that city. Yet for years now it always comes back to the Beatles and it is getting really, really dull.

*Disclaimer: I was born near Liverpool and my parents were both born in Liverpool and grew up there.

Thursday, July 03, 2008

An 'ambitious' programme...

The results of the programmatic review and how it will be implemented have been released; the BBC already has a story about it. I am sure STFC will be unhappy that the Beeb has focussed on the negatives given that:

STFC has balanced its budget and agreed on a very ambitious and scientifically sound programme of funding,
according to embattled STFC CEO Keith Mason.

Hmmm, 'ambitious', thanks to a certain 1980s political sitcom doesn't that word have some different connotations? It's right up there with courageous.

So what is my take of this document?

I think that one of the most controversial lines will be this:

In the case of the ILC related projects and BaBar, where support will be withdrawn, it is expected that any existing specific project grants will be phased out and the level of the existing rolling grants reduced accordingly.

This points to the threat of clawing back money on awarded grants - a controversial measure and a dangerous precedent. Depending on the levels of clawback this could result in a showdown with the university VCs who won't like the idea of this sort of instability in funding.

In the general comments:
We emphasise again that all the projects in the programmatic review represent good science. Given the need to introduce high priority new projects into the programme, we have had to reallocate funding.

As always this is welcome and works against the not-quite-a-whisper campaign against how 'good' certain science was as a non-public explanation for its non-funding.

The consultation panels expressed some strongly felt views on the programmatic review process that was carried out in 2007-8. We believe that the end result of the process is a very reasonable one... There are however lessons to be learned
I am not sure everyone will agree with the sentiments about the end result but the next bit is promising.

Advisory panels for PPAN and PALS will be set up as soon as practicable, and call for nominations for panel members in the PPAN areas is being made. We note the recommendation of Science Board that future programmatic reviews should be conducted with some kind of consultation process as the first step; this could perhaps be done by using the advisory panel inputs.
This is a step in the right direction and whoever decided to scrap the old PPARC advisory panels just before a CSR bid and the second programmatic review ought to have considered their position. Frankly, to think that new structures could be set up in time and to blame PPAN and PALS for not having done it is both ludicous and unworthy.

Kudos to science board for suggesting a first step consultation in the next review. Perhaps someone could also suggest tieing the review to the CSR timetable. Why review every two years on a a CSR cycle of three years? It seems silly to me and I am at a loss to understand why 2 years was chosen.

So what about STP?

Well in short Solar-Orbiter is okay (and PPAN considered it higher priority over other missions contrary to the advice of the solar and STP community as represented by the ad-hoc panel) subject to ESA still wanting to do it. But S-O is in the far future and there is a gap to fill until then and that is where the current stuff comes in.

STEREO PLS and HINODE PLS will continue as agreed having moved up the scale.

Cluster, SOHO PLS and the UKSSDC will all aslo continue as before. This is good news as Cluster is the only magnetospheric mission that we have at the moment.

STFC are now wondering how to take the issue of data-curation forward. They need to develop a strategy for dealing with long-term data sets so that they don't just vanish in the future. It seems to be left to chance at the moment.

BiSON is gone and STFC, like PPAN, have failed to address the weird double standard that has been applied here. There are plenty of small instruments and projects funded through the grants-line that were not included in the programmatic review; hence many projects escaped the review that otherwise should have been included if we take STFC at their word:

We note that while the consultation panel recommended that BiSON should not have been considered in the Programmatic Review, PPAN could not accept this recommendation, believing all projects should be evaluated. That is correct: the Programmatic Review is intended to cover all projects and facilities in the STFC science programme.
STFC need to deal with this major discrepancy and very soon. If smaller instruments funded by the grants line should be in the PR then put them in the PR otherwise, rely on the rolling nature of the grants line (and I don't mean rolling grants) to determine whether an instrument is still worth funding. As it stands, the current model means that some instruments have to jump two hurdles whereas others only have one. Bold and broad statements such as the one above, fail to adress this.

There is a hint of good news for ground-based STP. Although the vast majority of our infrastructure has been condemned to history this actually puts in words the issue that future ground-based STP proposals should be judged on merit and not based on a throwaway line in a strategy delivery document.

We intend to follow the panel’s recommendations. EISCAT is supported until 2011 and until this time proposals for its exploitation will be considered on their merits. Proposals for the support of ground-based STP will also be considered on their merits.

The key thing here is to ensure that we are mentioned in the upcoming strategy document as otherwise the key 'fit to strategy' criterion will always drop us down the rankings in the future. As for EISCAT, I think STFC has put itself into a very sticky situation with asserting its right to withdraw in 2011 based on a non-withdrawal letter. Their interpretation of the ad-hoc panel's view on this has caused upset:

The consultation panel accepted the inevitability of UK withdrawal from the EISCAT subscription in 2011,
I am told that the panel has sent a letter to Walter Gear (chair of PPAN) challenging this (amongst other points) and copied it to Peter Knight and John Womersley. What STFC do about this is up to them but if they just ignore it then I am concerned what that says about their methods of consultation at a time when they are finally making steps in the right direction.

It is still unclear to me how PPAN could have reached that conclusion and that erroneous interpretation has propagated to STFC. I hope that a correction and a public apology to the panel will be forthcoming otherwise it is yet another body-blow to community confidence.

Finally on a sligtly different note:

Looking ahead, many of the arguments made for the topicality of this area of research because of its links to climate and the environment suggest that it would be appropriate to involve additional funding partners. (It is interesting to note that ESFRI has classified the EISCAT-3D upgrade project as an environmental, rather than a physical sciences, facility.) We encourage the community to pursue other sources of funds, including the cross-council Living with Environmental Change programme, and we will seek to play an enabling role in any such discussions.
This is an interesting and promising statement. The STP community is already investigating avenues of cross-council funding as it is such a cross-disciplinary effort. However, this means that the community has to push this and not become apathetic about the whole thing. Past experience tells us that cross-council initiatives tend to fail as one council pushes things off to another with no one wanting responsibility.

This state of affairs will not change unless there is a tremendous push from the community. Pressue and constructive engagement needs to be maintained.

I'm back and with a bone to pick

...but not about the holiday, which was very nice. Found it harder to relax on this holiday than on previous, which is unsurprising given the current goings-on.

Anyway, the bone I wish to pick is with all those critics of the latest Indiana Jones film. Not necessarily the official critics (like Roger Ebert) who often quite liked it, but all those whiny people on the street and the internet who have been complaining how awful it is and how typical of Lucas. These are probably the same people who hated the Star Wars prequels to an irrational level (Yes I concede Jar Jar Binks is an annoying character but have you considered how irritating the droids actually are without the haze of nostalgia blurring your vision?).

For the casual reader beware, there are large spoilers ahead.

I saw Indiana Jones and the Kingdom of the Crystal Skulls on Tuesday night, long after most folks who care saw it.

I enjoyed it, perhaps not the same level of enjoyment as from Raiders or Last Crusade, but about as much as Temple of Doom (my least favourite).

It was very boys-own, pulp action story. I didn't even mind Indy surviving a nuclear explosion in a refridgerator. I got a little lump in my throat with the exceptionally affectionate references to Marcus Brody (the late, great Denholm Elliot) and Henry Jones Senior. I enjoyed the in-jokes that kept creeping in. I even remarked to my wife that Karen Allen still has a very nice bottom (assuming it wasn't a stunt woman in that scene).

I did not even mind about the McGuffin and the final reveal; it was faithful to the aura of the 1950s B-movie and action adventure story that it was trying to capture.

However, many so-called fans, have taken massive exception to this movie. There have been cries of too much CGI which might be justified. Although the prairie dogs and monkeys are clearly quite cool, they are hardly integral to the story.

What has really gotten my goat is the unbelievable amount of whining about the ending and the fact it involves aliens (or rather inter-dimensional beings) and that one of the characters claims to be psychic People are complaining that this is just silly and far-fetched.

That's right, the idea of aliens and psychics in an Indy film is silly and far-fetched.

Next thing you know they will have a man capable of pulling the beating heart from your chest while you can only watch in horror, and magical stones that lead to drought and starvation when removed from their place in a village.

Or, and imagine how silly this would be, there could be a 700 year old knight of the crusades guarding a magical cup that can heal mortal wounds and help you to live for ever as long as you stay hidden in a cave in the desert.

Better yet, what about a big box full of dust that can call down the power of God and melt people's faces? How stupid would that be?

So I have to wonder why none of these plit devices was considered silly, yet aliens hidden in a mysterious city of gold are? Is it just that we like our mythical mcguffins to be familiar to our culture? Is it an example that people are still wired to accept religious 'miracles' on some level even if they personally might scoff at the existence of a deity?

Who knows, but I wish they would just stop whining about it because I bloody well enjoyed it and I hope they make another one.