I think there are a lot of reasons why people might be interested in digital trials, that this is um digital trials. Allow us to reach populations that we wouldn’t uh. Otherwise they allow us to evaluate digital interventions and we’ll we’ll, get um deeper into some of the reasons for looking at digital trials. In terms of an outline for this presentation, first i’ll introduce a little bit more about digital trials and then we’ll talk about some of the advantages of digital trials. We’Ll talk about some of the disadvantages of digital trials and then i’ll introduce an example of a digital trial and go through some practical considerations and finally, we’ll end with a few open access. Resources so feel free to interrupt or or put any questions into. The chat box as we go along so um just by way of introduction digital trials refer to studies in which core elements of the trial process, so eligibility assessment, consent, allocation intervention, outcome, assessment and dissemination are digital and covet 19, as i mentioned before, has really driven Many trials towards digital methods – digital trials, include a broad range of things. This could be an in silico trial and that’s, using big data to estimate effectiveness, or it could be a decentralized testing trial. An m health intervention, there’s really a broad range of of digital trials, and so now i’d like to show a quick video, and this is basically um. This is from goldman sachs and it talks a little bit about what are some of the key um.

What are some of the key issues related to digital trials, so i’m going to screen share again? Can you guys see my? Can you see this? Yes, thanks. Okay, 80 percent of drugs fail in clinical trials and it can take up to 10 years to get a drug over the fda finish line and actually approve them. There is broader adoption of technologies like artificial intelligence. Like big data analytics, there could be a significant reduction in the overall cost of running a clinical trial and getting a drug approved Music. So we really thought about this into three distinct technology buckets. First remote, patient monitoring and cloud computing, not only from a convenience standpoint. Would also be because the protocol around the trial could be tweaked and adjusted as those results are monitored through the cloud. The second area that we looked at was around big data, analytics there’s millions of patient records out there in the world, and, if you think about leveraging those, you could better identify inclusion exclusion criteria around patients to more efficiently get enrollment for a clinical trial done. The last area that we looked at was artificial intelligence and within artificial intelligence. You can significantly reduce the administrative burden that falls on the physician in running a clinical trial. Another area that we think is emerging because of these technologies is in silico trials, so this would actually be where there’s no humans actually involved in a clinical trial, you’re just leveraging software and previous results from other individuals in order to make determinations around the efficacy of A drug so not surprisingly, given the size of the opportunity that exists within drug development and clinical trials.

Specifically, we have seen some of the larger tech companies start to pay more attention. Just given the innovation that exists within drug development, we would expect these companies to continue to play a major role. So so what are some of the advantages of digital trials? We talked about a few of these already reaching marginalized groups who don’t go to the clinic. The potential to reach rural individuals, people with disabilities and others that would less frequently attend a clinic. In addition, digital trials can evaluate digital interventions, so where the intervention itself is not tethered to the clinic is not tethered to an in person. Interaction – and this also creates new opportunities for co creation with the public and by co creation. I mean the researcher and the public working together to develop the intervention and then, as the as the goldman sachs video just pointed out, decentralized, remote monitoring. This is another opportunity and sort of an advantage of digital trials, so at the same time, there are some risks. Some problems, ethical issues, concerns feel free to write your answer in the chat box. What do you think of as some of the the problems of of digital trials, and how do we overcome these so um digi? The digital trials have many disadvantages that need to be considered. First there’s the digital divide that many people cannot access the internet and then even among those who do they might not be savvy enough to enroll.

In a study and some people, won’t use mobile phone applications, they won’t use digital interventions that are the focus of these kinds of trials. Many digital trials rely on self reported outcomes. Since you don’t have people going into clinics. The outcomes can sometimes be proxy ones. So these kinds of self reported outcomes need to be carefully examined and, and these self reported outcomes are, i think, a disadvantage of the field in general, digital interventions, won’t substitute for clinical services. So, at the end of the day, if someone’s tested for hiv, they would still need to go to a clinic and get tested in an official capacity and then there’s the potential for contamination that things can spread quickly through the internet and through social media. So if your digital intervention is assigning participants to different interventions, there is this uh inherent risk for contamination, and now what i’d like to do is kind of go through step by step and talk a little bit about a recent syphilis self testing digital trial that our Team organized for those of you who are interested in the results tune in tomorrow, dr wong chung will present the results in parallel session 5 at 3, 30, bangkok time. So this is just. This is more focused on the methods to kind of to to go through some key issues with digital trials, so the first component is study design and in this study we had three different arms, so people were randomized online to either get the standard of care, and This involves a wechat message: encouraging people to get tested at a facility based um testing center, so the facility they’d have they wouldn’t have to pay in money to um to get tested, but um.

They would have to figure out what time to go and show up and all those sorts of things and then the standard self testing arm had that same go to a facility, but also we mailed them syphilis self testing kits. So these are immunochromatographic um self tests. Where people draw their own blood, um and it’s a finger prick and then um using that finger prick blood, you can interpret the test result yourself, so you don’t have to send anything to the lab it’s a self test, and then the third arm was a lottery. Incentivized self test, so they still got the same self testing package. But then, in addition, they had a an opportunity to win a small amount of money um if they took the the syphilis self test kit. So in terms of consent and that’s, one of the first issues that comes up is how do you consent people online? Usually, this is sort of check boxes, it’s, very simple and sweet it’s, not the same as a written consent, and so irbs are going to look at this a little bit differently and for higher risk studies. You’Ll still need a written consent, so that’s an important consideration. You can often do video consent forms or have a video format to provide more detailed information, and then you should consider using captcha or similar tools to confirm that these are human participants. There are lots of different tools. Now we see that robots are expanding all over and um.

There are some it’s it’s important to confirm that your participants are are actually people and then, in terms of digital recruitment, social media recruitment strategies are, there are many different social media recruitment strategies. Each has its own source of bias that should be considered paid. Social media promotion can be useful in some contexts, and this is what raynor was referring to before for recruiting racial and ethnic minorities. This can be a boost, but it’s really not a panacea that you still need partnerships. You still need to think about these sources of bias. These are important considerations and then online panel samples and population based online samples or more rigorous ways to recruit people online, and you also need to think carefully about appropriate incentives, not too much not too little so goldilocks here and discuss with end users to get an Idea of what’s, reasonable remote monitoring is another key part of a digital trial. You can use self collection and with self collection that’s, where you take your own specimen and then you mail it to lab. Then the lab does. Does the processing they tell you the the test result so um, both self collection and self testing can provide interventions for remote monitoring and basically kits are sent to participants and then, in the case of self collection, they mail back the specimen and you can also have Photo verification systems to verify self testing outcomes where you take a photo on your phone and send that to a study staff via wechat, and you can see here a couple of qr codes from the syphilis self testing study that i mentioned in terms of retention.

Phone sharing in new telephone numbers can complicate phone based retention and there are a whole bunch of factors that can decrease retention rates i’m. Sure many of you are familiar with these mobility covered 19 concerns competing priorities, and this underlines the importance of engaging the end user group and really thinking carefully about participation from the end user perspective. Data analysis is largely similar to non digital trials. Digital trials do allow for rich analysis from computers and apps that you can get metadata. Sometimes um that’s, that’s, more detailed, the consent would form would need to d would need to ensure that the participant knows that all of this information is being collected and many online surveys will automatically capture things like ip address, so that’s an important component that all this An ip address is a very specific identifier and that’s going to be important to remove that kind of information and ethical review boards would want to see that and then there are also great opportunities for dissemination of digital trial data built in opportunities for engagement through social Media and other strategies – and you can consider co creating short videos or infographics to more broadly, engage with the public and then finally, preprints provide a nice way to share and to cite earlier so now, i’d like folks to think um, you can write in the chat Box, what are some creative digital ways to disseminate information about research findings? Um? So if anyone has some ideas, um feel free to put those in the chat box and and thinking about um creative ways.

Public engagement in research um, i think, co creation is a big. Is an important consideration here and co creation, meaning this iterative bi directional exchange between experts and lay people to design research processes and products um, and this can build trust. It can improve research quality and these infographics make your research findings more broadly understandable to the public. So this could be a public online call for input and i’ll describe one of those. It could be a crowdsourcing open call, as rainer mentioned a hackathon or a participatory design session, and thinking about sharing research findings online. There are a number of ways to sort of get your research findings out. More broadly, open access is increasingly important. Open access provides more exposure for your work. Um practitioners can apply your findings. You have end up having higher citation rates. Your research can influence policy. The public can more likely access some of your findings and many funders now require open access. So these are just a few considerations in terms of more broadly disseminating findings and then coming back to an online open call. This is an online open call that the editor in chief of the lancet richard horton, issued to nhs colleagues – and this is at the early during the early part of the covet epidemic. He said we need to collate more information about what the nhs needs um. Sorry, the slides keep on advancing and in the in both now and in the immediate future, dm me about your needs, so he collected all of these needs and then and then this ended up being included in a lancet article um.

So another really interesting digital way to um, to sort of co, create and disseminate online. So in terms of some open access resources that might be relevant, we have a plus medicine paper on co creation of infographics and videos, and this this provides practical tips and strategies. This came out last year and then the cherries checklist for online surveys. This is a checklist focused on online survey, reporting and research studies and there’s also a clinical trials transformation initiative. If you just google clinical trials transformation initiative, it provides some more information so i’d like to thank all of the sesh team members and a special thanks to dr tan and dr wu, who helped organize this digital.