info@biomedres.us
+1 (502) 904-2126
One Westbrook Corporate Center, Suite 300, Westchester, IL 60154, USA
Site Map
ISSN: 2574 -1241
ISSN: 2574 -1241
Received: August 02, 2022; Published: August 08, 2022
*Corresponding author: Klaus Hergan, Department of Radiology, University Hospital Salzburg, Paracelsus Medical University, Müllner Hauptstrasse 48, A-5020 Salzburg, Austria
DOI: 10.26717/BJSTR.2022.45.007222
The field of Artificial Intelligence (AI) is quickly growing and influences medical sciences, and Radiology in particular. Current literature is rarely dealing with the implementation process of an AI-Lab in opposite to the implementation and evaluation of AI-algorithms. This report describes perspectives how to install an AI-Lab within an academic hospital and for what it can be used. Many actions around the implementation process are necessary such as gaining permits from several institutional facilities, recruiting staff to run an AI-Lab, acquiring IT-hardware, and determining responsibilities. Access to radiological and medical data of patients is another complicated issue and must be consistent with patient safety and privacy as well as data security. Financial issues play a serious role in implementing and maintaining an AI-Lab. Rules are necessary for the communication inside and outside an AI-Lab. In conclusion, the implementation process of an AI-Lab is time-consuming with unexpected challenges confirmed by similar experiences in the current literature. For those, who intend to implement an AI-Lab, this report can be helpful as a basic support.
Keywords: Artificial Intelligence; Implementation; AI-Lab, Radiology; Computer Vision
Artificial intelligence (AI) is a very rapidly growing scientific and commercial field [1]. AI progressively targets medical Sciences in many ways [2-4], and Radiology in particular is predestined as a playground for data processing [5-8]. The number of scientific papers has been exploding during the last years [9]. Furthermore, the number of commercially available AI algorithms and of the related start-ups has recently skyrocketed [10]. As a Department of Radiology in an academic environment, the reasoning behind implementing an AI-Lab is threefold:
1) Basic research attached to a PhD program,
2) Access to enormous amount of medical data, which is normally not available to IT scientists, and
3) Independence from any commercial influence. The AI-Lab can be used in two main fields of activity:
a) The development of AI algorithms useful for patient management, and
b) The testing, as well as the implementation/evaluation of commercially available AI algorithms into routine workflow. Additionally, the AI-Lab can fulfil the function of assessing the possible use case of AI in everyday clinical practice as well as educating different professionals of the Department of Radiology in all matters concerning AI [11-13].
This report primarily focuses on the implementation of an AI-Lab for basic research and development of patient-oriented AI algorithms. So far, we did not encounter a publication on how to install an AI-Lab nor an instruction on how to run an AILab. Therefore, the purpose of this publication is to discuss the challenges, the successes, and difficulties concerning financial, technical, and employees’ related issues. Although this is a personal perspective, we undermine these experiences with the current literature. Furthermore, implementation of an AI-Lab differs in several points in comparison to other specific institutional constructional directions as we have prior experiences with installing other facilities such as a 3D-Lab or an interventional radiological simulation center, for instant.
Many things changed during the last decade concerning computed technology in Radiology: advancements in deep learningmethods, big data discussion, development of AI-algorithms, radiomics and other AI-activities, publications, and conferences on AI, as well as commercialization of AI. In addition, we experienced changes within and around the Department of Radiology like ongoing digitalization, contact to AI-technologies and companies, sub specialization within the department, increasing scientific interest in AI, as well as reservations about and interests for AI among radiologists. The idea to find an AI-Lab within the Department of Radiology, was supported by a long-lasting collaboration between a computer scientist of the University and a radiologist of the department of Radiology concerning AI-assisted segmentation of kidneys and renal cysts.
The rising interest in AI from the CEO of the University Hospital and the Dean of the Paracelsus Medical University, paired with the PhD Program of the University were considerable driving forces. The conviction to provide financial support was another driving factor. Furthermore, radiologists and radiology technologists expressed their interest in an AI-Lab within the Department. The decision to implement an AI-Lab within the Department of Radiology resulted in a chain of interdepending actions: approvals to start the project and to use money of the Department´s budget by the CEO of the University Hospital and by the economic director, the decision to connect the project with the PhD Program of the Paracelsus Medical University, recruiting PhD Students, PhD formalization and arrangement of a 3-years research plan, to decide for the type of technical equipment, and to find an Office for the AI Lab within the Department of Radiology.
Next to these basic requirements, we addressed considerations associated with AI: AI-equipment, storage space and data backup, connection to the local PACS, connection to the Radiology Information System (RIS) and to the Hospital Information System (HIS), data safety and privacy issues, machine learning algorithms, and responsibilities and related issues for tissue labeling. Many further considerations came up during the implementation process of the AI-Lab concerning the flow of information, regulations for employees, quality management, observation of the annotation process, data collection and presentation, statistical workup, and publication process. Integration of radiology technologists and radiologists of the Department was another important point and in which fields the AI-Lab could assist their daily workflow. As the activities around the AI-Lab is an ongoing process, further requests arise continually, which may be addressed in a future report.
We experienced many difficulties during the implementation process, which lasted for more than a year and is ongoing. The challenges we experienced are addressed in (Table 1).
As the federal government owns the University Hospital, the first step was to receive an approval to implement an AI-Lab. We sent a formless letter to the hospital CEO explaining the purpose of the AI-Lab and the collaboration with the Paracelsus Medical University. We did the same with the Dean of the Paracelsus Medical University, which is a private and separate institution. Both, the CEO and the Dean, who are in good standing with each other and impressed by that idea, gave their approval.
Table 1: AI-Lab Implementation Process: necessary action, unexpected difficulties, and possible solution.
The 3-year postgraduate PhD-program is an institution of the Paracelsus Medical University including lectures, presentations, research, and publication. A formal application has to contain the purpose of the scientific program, a 3-year research plan, the head of the program and other responsible people, and the funding of the tuition fee.
As PhD-student, we recruited the IT-scientist to represent the technical part of the AI-Lab, and who is employed in a large ITCenter and responsible for AI there. A medical doctor, who just completed his medical university degree at the Paracelsus Medical University, applied for the PhD-program as the medical part of the AI-Lab. We had prior experiences with this applicant in kind of a scientific collaboration. Both applicants received permission for the PhD-program, the IT-scientist extra-occupational, and the medical doctor in full-time. Both variants are possible in this PhD-program with no difference concerning the ECTS-Points and timeframe. The Department of Radiology entered into a collaboration contract with both students and decided to finance the University fee for both students, as well as the salary for the full-time student. Much paperwork was a challenge for that part during the implementation process of the AI-Lab.
We found a room within the department of radiology suitable as an office for the AI-Lab. This room needed to be adapted to be suitable for the IT-equipment and as a workplace for at least one person. Planning and organizing the renovation was without any problems. Although self-evident, the demand for and installation of an air-condition system took several months. In the meantime, we needed portable cooling systems due to a possible fire hazard of an overheating workstation. Unfortunately, the installed air-condition system is not connected to the central technical monitoring to automatically trigger an alarm in case of malfunction. Until a better solution is found, users who can access and work on the systems via the general VMware Horizon desktop-sharing system do monitor the GPU temperatures manually. If there are signs of temperature overload, the users can shut down all workstations immediately
The IT-specialist decided for a DGX-Station [NVIDIA], and a second, dedicated GPU-workstation. The DGX-Station is a high-end AI-workstation from Nvidia Corp. The specifications are as follows: 1 Intel Xeon CPU, 3.2 GHz, 28 cores, 256 GB RAM, 4 Nvidia Tesla GPUs [Volta generation], 4 SSDs (3 SSDs operated in RAID0, 1 SSD system+ home directories), 10 Gb/s Ethernet. The DGX-Station offers high-speed interconnects (NVLINK) between GPUs, making it especially suitable for multi-gpu computations. The dedicated GPU-workstation has been provided by SysGen GmbH, Bremen, Germany. It is based on 1 AMD Threadripper CPU, 3.2 GHz, 24 cores, 128 GB RAM, 2 Nvidia RTX 3090 GPUs (Ampere generation)., 1 SSD (2 TB, system and home directories), 2x 14 TB hard drives (aggregated into one ZFS volume), 10 Gb/s Ethernet.
We received permission to use money out of different scientific funds of the department by several authorities, and by the radiologists who decide on the use of the university fund together with the Chair of the Department of Radiology and the economic director. The Radiologists endorsed the AI-Lab arguing that investing into such a project is preferable than in other projects or into the hospital´s budget.
Before starting annotation of data from the PACS several preliminary steps were necessary since the European data protection laws and regulations changed the national laws for data management completely. It is a very complex and iterative process to get access to patient data for research purposes. We were faced with a lot of resistance from the Hospital´s IT-department. Although it is an academic facility, the Hospital´s IT-department is neither personnel nor professionally equipped to meet requirements and needs for research. With personal engagement of both parties, we received permission to connect the workstations to the PACS, to retrieve patient data in selected cases from the RIS and HIS, and a hospital´s lawyer approved the project’s data safety, admittedly with the prerequisite that the governmental ethics committee approves the project. For any planned data retrieval, we have to provide the IT-department with a list of specified patients to receive the data transfer permission. This approach should guarantee that patient´s consent or dissent will be respected.
Although we only described the hypothesis of the AI-Lab, since we had no access to the hospital´s data sources at the time of application, the Ethics Committee met a positive decision, as we only analyze data retrospectively. The process of applying for an approval of the ethics committee would have been much more complicated if we were interested in prospective studies or in inventing and testing self-made AI algorithm as a medical product.
This report does not intend to describe the development of AIAlgorithms, the annotation of radiological images, or the outcome of machine and deep learning. Furthermore, the report does not address the relationship between radiologists and clinicians concerning the outcome and possible biases of developed AI algorithms. Instead, we like to report how the AI-Lab runs and how people interact with each other. As the AI project started during the Covid-19 pandemic, we intended to install a remote access for transferring work to home offices. This took many months because of security concerns. In order to keep each other and the department updated on the current progress of the AI-lab we introduced an AIJour- fixe, which is held both onsite and online once every month. The goal is to implement a periodic consultation hour for Radiologists and Technicians of the Department to attract people for AI, develop ideas for use cases of AI, and mainly to improve acceptance and interest in AI. Additionally, we organized presentations and held lectures for the employees of the department, to inform them about AI and the AI-Lab. We are planning to organize a further education for the radiological community of the county.
Just at the point of preparing publications for Journals, the medical PhD student quit his job, discontinued his studies at the University, and switched career paths completely out of personal reasons. This was unexpected at a time when the AI-Lab started to run successfully. This in turn meant a personal and financial setback for the project. We went on to advertise the AI-Lab within and outside of the department and found two new PhD students in the meantime. It is encouraging, that the AI-Lab attracts students from the Paracelsus Medical University to be part of the team, to help in the annotation process, and to support us with scientific publication. These students are diligent, helpful, and resourceful and possible PhD students as well as residents for the future.
AI is becoming a set part of our routine radiological world [14]. A critical view is necessary in order to weigh the advantages and disadvantages of AI algorithms [15]. Optimism for and pessimism against AI are daily companions in the radiology community [16]. Publications based on AI are growing rapidly and with that, radiologists have become more familiar with definitions of AI terms [17]. In addition, the demand for radiology increases with the growing complexity of therapeutic options leading to a workload, that conventional Radiologist will not be able to handle in the future. Considering this fact, AI companies grow quickly, develop AI algorithms, and computed assisted diagnosis systems (CAD) continuously to cater for a growing demand. AI solutions promise to save time and money, increase diagnostic accuracy and quality, and to take on a supporting role for clinicians. What radiology currently really needs is the question, how to use the fast-developing opportunities to achieve the goals mentioned above [18]. How we evaluate AI companies, and their AI algorithms is another issue [19]. Leiner, et al. suggest developing a platform within healthcare systems for the deployment of vendor-independent AI solutions [20].
Keeping all that in mind, we decided to implement an AI-Lab within the Department of Radiology in a University Hospital. The main contributing factors were a preexisting collaboration between a radiologist and an IT-scientist focused on the development of an AI algorithm for kidneys and cystic renal disease, as well as a PhD program offered by the Paracelsus Medical University. The main goal of our AI-lab is vendor-independent basic research on AI algorithms. The secondary goal is to evaluate commercially available AI solutions. Several publications describe the challenges, the advantages and disadvantages, and the costs and possible financial benefits addressed by the AI companies [21-23]. On the other hand, there exist only few reports on the challenges to implement an AI-Lab [24]. Several publications discuss the challenges of implementing AI applications [25-29]. The main reasons for a hesitant implementation of AI are insufficient and inconsistent performance of AI systems, problems during the implementation process, uncertainty about the value of AI algorithms, cautious acceptance and trust on both sides [radiologists and referring clinicians], mental reservations from health care providers/ stakeholders, patient´s considerations concerning privacy and data safety, security issues and IT-resources in healthcare systems, and many more.
The results of our report lighten up the problems during the implementation of an AI-Lab with its primary goal of basic AI research. First and foremost, important are the people, who have the idea to implement an AI-Lab. The implementation process can fail quickly if one member of the team gives up out of frustration. If that happens the supporting parties are responsible to cool down the situation, come back to the roots of the enterprise, select somebody fitting into the team, and convey hope.
Second, the many permissions needed for the implementation process of an AI-Lab demand patience and time by the AI-team. It depends in which organization someone is working in, and who is lastly in charge of the final decision. Within an academic center like ours, the hospital´s CEO is liable for everything at the end. The CEO is the only one who reports to the county´s government, which is the owner of the hospital and finances most of the expenses. The medical director is responsible for the well-being of the patients and in a broader sense for everything happening around the patient. The Dean of the Paracelsus Medical University is interested in secure, reproducible, and honest research among other things. The local legal committee considers legal issues of scientific projects and gives recommendations to the project manager and the medical director. Up to now, we do not have a local ethics committee. Therefore, all scientific projects need to be approved by the regional ethics committee installed and financed by the county government. All of the aforementioned authorities have their purpose, duties, and responsibilities. Understanding the decision process in advance helps to save time and nerves.
Third, the choice of computer hardware is a very essential step during the implementation of an AI-Lab. IT-equipment is the beating heart of an AI-Lab. It is impossible to make the right choice and run complex AI-tests without a dedicated specialist. It is very unlikely to find such a person within the IT-department of the hospital. Such a specialist must have an education in AI in order to be familiar with the complex machine learning processes and in creating AI-algorithms using neural networks. In addition, this person must speak a language understood by the AI-Lab partners and the local IT-department, as countless interactions come up, and need solutions at eye level between all people involved in the implementation process. Fourth, organizing the infrastructure of the AI-Lab is another important issue. Computers must stand in a room with air-conditioning, which, if possible, has a warning system in case of malfunction. Cooling is a very serious issue as computers heat up enormously during processing. People working in an AI Lab need a workspace on site, but also a remote access from a home office to be able to achieve the required productivity.
Fifth, the basic activity in the AI-Lab starts when everything is done and is running. Research depends on people and the idea. IT-technician or physicist specialized on AI are responsible for programming dedicated computers, whereas the medical counterpart is trained to interpret radiological images. Both parties must work together as partners. Computers only work on data fed into the system and only learn on the tasks programed into it. Whatever task a computer should do is in dependency on reliable and huge volumes of data. Therefore, you need unlimited access to data, many helping hands for labeling the data, and control mechanisms on reliability and safety issues. With all these factors in place, it is possible to work on reproducible AI-algorithms and CAD systems. Moreover, most important is to respect patient´s consent or dissent concerning data usage, to find a correct and clear way for data retrieval according to the European data and the national data protection laws, which may differ between countries. Sixth, people working in an AI-Lab dedicated to basic research should be interested in sharing their knowledge. A PhD-program of an associated University may be helpful. It is also essential to have critical discussions and reviews for further development within the AI-Lab and AI-program.
Seventh, you need money in order to finance the project. The idea that AI helps to reduce expenses in healthcare systems is a misconception, at least today. Primarily, implementation and running of an AI-Lab for basic research and even for evaluation of commercial AI-products cost a lot of money. Moreover, nobody knows if there will ever be a revenue on investment. Most of the non-commercial AI-Labs in public and private healthcare facilities probably are financed indirectly. If an AI-Lab successfully creates AIalgorithms with an added value for patient management, indirect profitability may be reached with self-promotion or founding a spin-off company for commercialization. Up until now, AI-Labs within healthcare facilities have tried to improve the quality and safety in patient management with manageability of the invested money.
Although we have experience implementing institutional facilities for scientific, quality assurance, patient care, and teaching purposes and although every project varies from each other, the installation of an AI-Lab is very different in some points. Handling data management and data security regulations are serious issues. The need for IT-scientists specialized in AI as a new profession within hospitals und departments of radiology is another point. Medico-legal, regulatory, and in particular psychological issues are multi-layered and dynamic and not the same what we have seen in other institutional projects [30]. As this report deals with the experiences of a single center, we would like to address some weaknesses. There is nothing measurable and comparable, which is an eminent principle in science. Some of the cited publications describe the implementation and/or evaluation of commercial AI systems, the challenges and successes, the costs and benefits, the problems and hopes. As we did not find something comparable within the current literature, we encourage others to share their experience in this field. Furthermore, we were convinced of the idea of implementing an AI-Lab for basic research, but in reality, we had no detailed instructional manual. This leads to delay and frustration, which has to be compensated by enthusiasm and hope. Moreover, this report does not include a framework how radiologists and clinicians interact with each other concerning the strengths and weaknesses of AI algorithms in respect to patient care.
Implementing and running an AI-Lab is a very complicated, time, nerves, and money-consuming process. On the other hand, the gained experience helps in many ways to evaluate critically the growing world of AI. Firstly, the dedication of an AI-Lab within a healthcare facility should be evident - basic research, evaluation of commercially available AI systems, or both. Secondly, planning the steps for the implementation of an AI-Lab is essential in order to receive necessary permissions, to choose the correct people and equipment, to get access to data, to guarantee security regulations with the processed data, to report results to the public, and possibly to commercialize the developed AI-algorithms. We recommend thinking about the necessary steps for implementation of an AI-Lab in advance and to work on these steps in parallel to avoid loss of time. We want to encourage radiologists and physicians of other medical specialties to install and run an AI-Lab or to find access to an existing AI-Lab within a healthcare facility to become familiar with, to develop a critical view about, and to work with AI for improving quality and safety of patient management. As Rodriguez et al concluded: AI will not replace doctors and researchers, but clinicians who use AI will replace clinicians who do not [31].
The authors contribute there is no conflict of interest, no financial support, and no otherwise funding.
Abstract
PDF
