The Covid-19 pandemic advanced the era of digital healthcare, but the medical community should have a critical point of view to verify the potential and actual effect of digital health, an expert said.

Harlan Krumholz, director of the Center for Outcomes Research and Evaluation (CORE) at Yale University, discussed the medical community’s role in introducing digital health products in the “Next Normal & Digital Healthcare in the Post-COVID-19 Era” session at Korea Healthcare Congress 2021, hosted by the Korean Hospital Association on Thursday.

Citing a Korean idiom, “don’t drink the kimchi soup first,” Krumholz said doctors should not hastily change treatment methods until they can see digital products prove their ability to improve treatment.

“Digital health technologies are certainly necessary, but simply having technologies does not guarantee the improvement of the outcome of clinical intervention,” he said.

With growing interest in digital health in the wake of Covid-19, the U.S. government revised policies to promote telehealth. The FDA developed the criteria to evaluate software as medical devices (SaMD).

However, not all digital health products are subject to regulation or verification, and insufficient data for verification makes it difficult to confirm the effect of digital health products, Krumholz said.

Data used for analysis and clinical validation are not disclosed, regardless of whether they are regulated or not, he went on to say. Even if the data is controlled, essential information such as whether the clinical trial was conducted on all populations is often missing, he said.

Krumholz predicted that publication bias, only to publish results that show a favorable finding, would occur more frequently in the digital health sector.

A five-year investigation revealed that one study supported by the U.S. National Institutes of Health omitted one-third of the results because of publication bias, he said.

Clinical studies testing remote products can face the problem of participants leaving the designated site in the middle of the study. If researchers derive outcomes out of the few remaining participants at the end of the trial, there is a risk that the results could be biased.

Even when doctors directly encourage patients to participate in clinical trials, about half of them only use the product for about a month, Krumholz said. “We should enhance the criteria so that we can examine and evaluate the product in real settings,” he said.

Krumholz proposed making an innovative ecosystem that requires digital health companies to be responsible for the effect of their products. He said the starting point would be asking a fundamental question of what the patient gets from using the product.

“If doctors are swayed by the surge of digital health products and the market trends, they might choose a useless or even a harmful product. They should approach this issue fundamentally,” Krumholz said. In product evaluation, doctors should ask whether the patient’s life has improved, whether the survival rate, quality of life, and cost of care have improved, whether the patient had an opportunity to achieve health goals, and whether the patient’s preferences and values were respected, he added.

He also urged caution against unexpected side effects of digital health products.

He said every intervention has to evaluate the possibility of causing unexpected adverse reactions. There is always a possibility of erroneous results on a changed system, even with good intentions.

“Doctors should focus not only on expected benefits but the potential for harm. The harm may be bigger than the intended benefit,” he said.

Lastly, he said the medical community should reward the best treatment results, not the salesperson, in an era of great digital health promises. “The medical community should have a critical view on the product’s all lifecycle and demand evidence,” he added.