Background: Digital health applications (DiGAs) increasingly offer diagnostic and therapeutic functions. The so-called "symptom-checkers" are a group of DiGAs with diagnostic algorithms. However, for many of the available DiGAs, the additional medical or organisational benefit is currently unclear. Based on a project by the AIHTA ("Framework for reimbursement decisions of digital health applications", published in 2020), the outlined evaluation process was used in the assessment of a symptom-checker. After this evaluation, and considering legal as well as technology-specific aspects, the proposed evaluation process for DiGAs was further adapted.

Methods: DiGAs from directories and lists from four countries were compiled into a meta-directory. Expert interviews were conducted to identify DiGAs from Austria, to establish prioritisation criteria, and to gather information on technology-specific requirements of DiGAs. To analyse the evidence on symptom-checkers, a report from the British National Institute for Health Research (published in 2019) was updated. Finally, the outlined evaluation process for the evidence-based assessment of DiGAs was adapted by adding additional evaluation steps to the proposed design.

Results: 132 DiGAs were collected in the meta-directory. Of these, 38 DiGAs currently fulfill the prioritisation criteria (the DiGA offers therapeutic and monitoring functions for chronic disease management programmes, the DiGA is available in German). For the majority of the 38 prioritised DiGAs, information on CE marking (n=29) and data protection (n=35) is available; the technical requirements for an interoperability with the Austrian electronic health record (ELGA) are fulfilled by seven DiGAs.

As for the evidence on symptom-checkers, 27 studies of the systematic review by the British National Institute for Health Research were updated with 14 additional studies. For the diagnostic accuracy (ten observational studies) and the accuracy of triage disposition (eleven observational studies), the studies did not provide satisfactory results due to a high risk of bias. At the same time, there are concerns regarding misdiagnosis and overdiagnosis.

Conclusion: For the evidence-based assessment for reimbursement decisions on DiGAs, the collaboration of different Austrian institutions is recommended (in particular for the evaluation of data protection and technical interoperability requirements). The prioritisation criteria to filter relevant DiGAs are based on the legal implications of the General Social Security Act. For symptom-checkers, these prioritisation criteria are not fulfilled consistently. The analysis of the evidence showed that for symptom-checkers there is currently insufficient evidence to show a medical or organisational benefit.

Based on this exemplary evaluation, the AIHTA recommends a process design for the evaluation of DiGAs that combines prioritisation criteria with regulatory, technology-specific as well as evidence requirements.