The results are in from the Reach2HD study, which was designed to test the experimental drug PBT2 for early and mid-stage Huntington’s disease. The drug seems safe and well-tolerated at the doses that were tested, but we have major concerns about the way the results have been reported.

What is PBT2?

Prana Biotechnology, an Australian drug development company, is working to develop a drug called PBT2 for use in Huntington’s Disease and Alzheimer’s Disease (AD). The company has previously reported that the drug has positive effects in animal models of HD, and that it is well-tolerated when given to human research participants.

The drug works in an unusual way, which is to reduce interaction between the HD-causing huntingtin protein and the metal copper in the brain. Copper, in small and well-regulated amounts, is critical for the normal function of cells. In fact, copper is important for our cells to produce energy, so without it we’d have a hard time!

But in diseases like HD and AD, metals like copper can start to have harmful as well as useful properties. Some scientists think this may contribute to the early death of cells in the brains of patients with these diseases.

Prana tested PBT2 in mouse and worm models of HD, and found that it led to improvements in some signs of disease in these animals.

What is Reach2HD?

Given favorable results in the lab with animal models of Huntington’s disease, Prana Biotechnology decided to test PBT2 in people with HD. They worked with clinical centers under the direction of the Huntington Study Group in the US and Australia to run a trial they called Reach2HD.

The Reach2HD trial involved 109 Huntington’s disease patients with early or mid-stage disease who participated for about 6 months. During that time, they were randomly assigned to one of three groups: a low dose of PBT2, a higher dose of PBT2 or a ‘placebo’ group, who received dummy pills containing no drug. Neither the patients, nor the researchers running the study, knew who was receiving active drug and who was assigned to the placebo group.

This kind of trial — called by researchers a randomized, double-blinded, placebo-controlled trial — is the gold standard for testing new drugs. At two timepoints — 3 and 6 months — all of the participants were given a battery of tests, including brain scans and blood sampling.

Prana’s announcement

In a press release, Prana announced what sound like very impressive results from the study. They say PBT2 was “safe and well-tolerated”, “met its primary endpoint”, produced a “significant benefit on cognition” and brain scan changes “suggestive of a beneficial effect”.

That sounds amazing… almost too good to be true, in fact.

First let’s remind ourselves that the claims were made in a press release, not in a peer-reviewed scientific publication. That means the company’s claims have not yet been subject to the level of scrutiny needed to be accepted by the scientific community.

Now let’s look at what each of the claims means in practise. First we need to learn a bit about clinical trials.

A Phase 2 trial

Regulatory agencies like the FDA need various kinds of evidence in favor of a new therapy, before they can approve it for patients. First, they need to ensure the drug is generally safe in people, having been tested previouslt in animal models. This is established in what’s called a phase 1 study, in which a few volunteers take doses of the drug under closesupervision, to make sure there aren’t unexpected side effects.

Once we’ve established that the drug isn’t highly toxic, we’re interested in whether the drug is safe for patients, and whether it works. So-called phase 2 studies are designed to further establish whether a drug is safe — that it doesn’t make the disease worse, for example. They also aim to get an idea of whether a drug might be effective.

Finally, a drug with a successful phase 2 study can be tested in a larger group of patients to confirm the beneficial effects seen in the smaller phase 2 study. These phase 3 studies are usually the basis of an application to have a drug approved and sold to patients.

Reach2HD was a phase 2 trial — intended to establish the drug was safe, and to get an early idea of whether it might have some benefits. So Prana’s statement that PBT2 “met its primary endpoint” simply means the drug was safe and well-tolerated.

Why we need endpoints

Testing whether a drug is 'effective’ or not can be challenging. In Huntington’s disease, lots of things go wrong. Patients experience movement problems, thinking and memory problems, depression, apathy, brain shrinkage, and a whole host of difficulties with the activities of daily life. Which of these symptoms should we target if we want to fight HD?

Complicating things further, there are often dozens of different ways of measuring a particular feature. For instance, there are many ways of testing thinking skills or ‘cognition’ in HD. Which measurement should be the one we use to decide whether the drug is working?

The goals that a drug has to meet in a trial are called endpoints. A very important feature of Phase 2 and 3 studies is that the endpoints need to be established in advance. The goalposts mustn’t move after the trial begins. Otherwise, nobody will be able to agree on whether the trial was a success.

Reach2HD had safety and tolerability as its primary endpoint. And indeed, the drug didn’t cause too many side effects, and not too many people stopped taking the drug.

One patient who took the higher dose experienced a worsening of their HD symptoms after finishing the study. Though this happened after the patient stopped taking the drug, the investigators decided that the effect was due to PBT2, which raises an important caution for future studies.

But overall, the safety and tolerability news is good for PBT2.

Secondary endpoints

In a phase 2 clinical trial, many different things will be measured, to give a broad picture of what aspects of the disease are affected by the drug. The measurements the trial designers think are important, are called secondary endpoints. Again, these are set in advance to avoid confusion later.

Reach2HD had seven secondary endpoints: cognition; motor function; functional ability; behavior; global assessments (whether people are feeling better overall); blood and urine tests; and brain scans.

What’s more, each of these endpoints was based on many different individual tests. For example, the cognitive endpoint contained eight different thinking tests.

So when the press release claims success in meeting the cognitive endpoint, you might think that drug-treated patients had improved on all eight tests… or maybe four out of eight?

Unfortunately that’s not what happened. There was only a ‘significant’ difference on one of the eight tests - called ‘trail making B’. That involves connecting letters and numbers with a pencil against the clock. None of the other tests was better in drug-treated volunteers.

So, while it may be technically correct for Prana to claim that a cognitive endpoint was met, cooler minds will want to look behind the headline, and consider the seven tests that did not improve.

Multiple comparisons

Humans are optimistic by nature — and HD family members are desperate for good news. But it’s generally frowned upon in the scientific community to report only positive findings, or to give them undue prominence. That’s because of what we call the problem of multiple comparisons.

To understand this, think about flipping a coin. If you flipped a coin ten times and got ten heads, you’d wonder how honest that coin was! But, if you flipped a coin a million times, you’d expect to get ten heads in a row several times, somewhere in the million flips.

Put simply: the more things you test, the more likely one will show positive results, simply by chance. That’s why we’re skeptical about the single cognitive test that improved among the eight that were tested.

“The more things you test, the more likely one will show positive results, simply by chance. That’s why we’re skeptical about the single cognitive test that imoroved among the eight that were tested ”

In the Reach2HD study, the investigators cast a very wide net — measuring 7 different categories of HD problems, each measured with multiple different tests, ultimately measuring dozens of different things in all 3 groups of people.

In fact, almost all of the tests for the secondary endpoints were not improved by the drug.

From the limited data released by Prana so far, it’s not clear how they dealt with the problem of multiple comparisons. Remember, so far we only have a press release: the full results haven’t been peer-reviewed and published.

There are well-established mathematical techniques for dealing with the problem, but they’re not always used correctly, and it’s not clear exactly how Prana dealt with this problem in their clinical trial analysis. The press release doesn’t say — but this is really important stuff.

What about the functional improvement?

Prana’s press release claims PBT2 was “associated with a favourable signal in functional capacity.” Surely that’s good news?

Doctors use the word ‘function’ to refer to how people are getting on in their everyday lives. Things like working, household chores, finance and so on. This is different from the cognitive testing we’ve already mentioned. Huntington’s disease causes a steady decline in function, and there are a number of well-established ways to measure this, turning function into a number that can be compared across groups.

Reach2HD used two different measures of function, as well as two measures of ‘global’ wellbeing which are closely related to function.

So what was this “favorable signal in functional capacity”?

What was actually seen was a small difference in one score — the total functional capacity — in patients receiving the higher dose of drug, compared to placebo-treated volunteers.

Scientists use statistical tests to help decide whether a difference like this is real or arose by chance. If the difference is big enough, it passes the test. If it’s not, it fails and the scientists are not allowed to say there was a ‘significant’ difference.

The difference in functional capacity scores in Reach2HD came close, but did not pass the statistical test. That’s why the phrase “favorable signal” was used instead of “significant difference”.

That turn of phrase might be technically correct, but we don’t think it’s helpful in conveying results accurately to HD patients and families.

What’s more, the other function score, and the two scores reflecting global wellbeing, didn’t differ between groups.

It could be argued that the responsible thing to do here, given the desperation of HD families for good news, would be to say “no overall functional improvement was seen”.

Instead Prana chose to trumpet the borderline positives while playing down the negatives.

But the brain scans!

The final claim we want to look at is that PBT2 “reduced atrophy of brain tissue in areas affected in Huntington disease”.

Some Reach2HD volunteers had two brain scans to measure the brain shrinkage, or ‘atrophy’, that HD patients experience. Reduced atrophy sounds great! But what did they actually show?

Believe it or not, the atrophy results described in the press release are only based on the brain scans of two patients from each group!

We’re genuinely amazed that this analysis was done in such a small number of volunteers. And we’re even more amazed that Prana has chosen to report this as a positive finding from the study.

Two in people from each group is nowhere near enough people to even begin to understand what’s happening with atrophy. It typically takes dozens of volunteers to be able to even detect brain shrinkage due to HD over six months, let alone measure the tiny difference a drug might be making.

The claim that brain atrophy was reduced by PBT2 is clearly unsupported by the described data. We’ll have to wait for an analysis of the entire data set before seeing whether this claim is actually true.

This bold claim is another reason why we view the press release with skepticism and some disappointment.

Take home and next steps

We think two conclusions can safely be drawn from the Reach2HD press release.

First, that the drug is safe enough to proceed to larger trials.

Second, that the claims of cognitive, functional and imaging benefits are not supported by enough evidence to place much confidence in them.

We’re as enthusiastic as anyone for drugs to benefit HD patients. Your humble author, for instance, is an HD researcher and research volunteer who happens to carry the HD mutation himself.

To be clear: we’re not saying the Reach2HD press release contains untruths. But, in our opinion, it does contain statements that overemphasise the positives in the trial’s results, play down the negatives, and are likely cause false hope in HD families.

We’re all for hope, but we’d rather have cautious optimism than hype and false hope.

We’re also keen to see a larger, phase 3 trial of PBT2. But first, we call on Prana and the HSG to submit the trial results to the proper scientific scrutiny of a peer-reviewed publication so that researchers and HD family members can see and evaluate the full data set.

Meanwhile, we recommend that HDBuzz readers arm themselves for reading future press releases with our Ten Golden Rules for Reading a Scientific News Story.

Share