A NYT article on a woman who compared the results of 3 genetic test results. The article focuses on the wild west nature of genetic testing and interpreting the results that get sent back.

Here’s an excerpt:

In the case of Type 2 diabetes, inconsistencies on a semantic level masked similarities in the numbers. G.T.L. said my risk was “medium” at 10.3 percent, but 23andMe said my risk was “decreased” at 15.7 percent. In fact, both companies had calculated my odds to be roughly three-quarters of the average, but they used slightly different averages — and very different words — to interpret the numbers. In isolation, the first would have left me worried; the second, relieved.

The problem is not necessarily that the tests are imprecise, though any or all of them certainly might be*. The problem is most likely a bit of accuracy and lot of interpretation. I just want to focus on the interpretation question, in part, because that is the issue that has led the FDA to essentially ban direct-to-consumer testing. The language that the companies use to convey the results to the customer is the interpretation. Unlike with most medical tests, the consumer sees the results directly and without the necessary intervention of a doctor. I think this is a good thing. But it does mean that the company, not a doctor, must describe the results. Again, I think this is a good thing – companies are in the business of serving their customers, whereas it’s not entirely clear who doctors serve (that is, it’s not entirely clear who a doctor sees as their customer). As such, companies are virtually guaranteed to exhibit better customer service than doctors. However, it’s not what customers – medical patients in this case – are accustomed to.

An similar example is a blood panel. A typical blood panel will have about 20-50 numbers on it. Patients often don’t see the results but are told an interpretation of the results by a doctor. If you do, however, look at the results, you see that whatever interpretation the doctor gave you is probably more exact than it should be. Every number on a blood panel will be accompanied by a “normal” range or some indication of what very high or very low values are for that number. A doctor will do the interpreting for you, but if you look at the numbers, you can do it yourself. You just don’t really know what to make of most of them. For example, suppose a “normal” range is 100-500. I’m looking at a test right now and see such a range. This is a huge range. One way to look at it, the way I look at it, is that the average value is 300 with standard deviation (at what level of precision I have no idea) of 200. That’s nearly 100% “error” in the measurement that constitutes normal. If the result is near 300, say 250 – 350, then I’m going to interpret that as normal. If, however, I score a 110, am I still normal? Or am I close enough to the end of the range that I’m actually at some kind of risk? But then, if so, what does 90 mean? It’s the same distance from the end of the range, but actually does fall out of the range.

This will all get sorted out in time. Given the draconian nature of health regulation, however, I’m confident that it will get sorted out by preventing companies from doing the interpreting and forcing doctors to do it. I think this means more of the same old same old for medicine and means less innovation and change by companies working on next gen products.

-JD Cross

* – This article doesn’t address the precision of any one particular test. To do that would, however, be quite simple: order 10 or 20 or 30 tests from the same company and see if the results match. This is the test that I would do first, before doing the “test” the NYT writer describes. That is: before I try to compare any test for accuracy, I would want to know if any test is precise.

 

Advertisements