1.2.1.

General pre-MRI screening procedures

Accordingly, at an initial patient visit for an MRI examination, knowledge of an LLIED previously inserted at another institution is typically gained through direct interaction between the scanning supervisors (i.e., physician or technologist) and the patient (hopefully, possessing specific LLIED details). This is followed by manual entry of attained screening information into the patient’s electronic medical record (EMR).^6,12,13 On the other hand, if the LLIED was intramurally placed, this information is likely gleaned by EMR review. Regardless, both forms of data extraction and documentation have known deficiencies for safety screening,¹⁵ thereby enabling an LLIED to remain inadequately recognized up to the time of (and possibly during) MRI scanning, especially in stressful emergency or trauma situations.^16,17 Compounding LLIED-specific potential risks from MRI exposures are unpredictable causative factors related to patient or scanning differences.^4,6,9,12 Other scenarios where LLIED recognition is also important include (1) External cardioversion (potential device malfunction and/or damage); (2) Radiation therapy (potential device malfunction and/or damage); (3) Cremation (potential device battery explosion).^18,19

As mentioned, prerequisite patient and/or LLIED assessment or preparation (before, during, or after an MRI examination) may differ even when LLIEDs are considered MRI conditional. For example, when pertaining to an MRI-conditional lead-less pacemaker (LLP), the expectations typically include cardiologist-dependent (1) Pre-MRI evaluation of the patient and/or LLIED (likely necessitating LLP-setting adjustment); (2) Direct patient monitoring during MRI scanning; (3) Post-MRI evaluation of the patient and/or LLIED (with LLP resetting to original state).^6,12,13 These demands exceed those when an MRI-conditional lead-less recorder (LLR) is involved, and precautions taken alone by the MRI technologist are deemed adequate.^6,12,13 Therefore, the failure to differentiate between these two common MRI-conditional LLIED categories (i.e., “assessment-requiring” and “simple,” respectively) well before initiating MRI scanning could either put a patient at undue risk or disrupt operations (e.g., incorrect pre-examination readiness of supporting services, such as cardiology).

1.2.2.

Use of a chest X-ray in pre-MRI screening

A chest X-ray (CXR) is a standard component of pre-MRI safety screening (for LLIEDs or other man-made objects in the chest).^20–27 Such CXR-based screening assumes even greater importance when there is inadequate EMR documentation from lack of a prior visit and/or internal misrecording.^26–28 Unfortunately, any LLIED could be overlooked on a CXR due to their mutually small sizes (subject to projection-related distortions), especially when accompanied by (1) Suboptimal radiographic technique (e.g., under-penetration); (2) Patient-related factors (e.g., motion-related blurring); (3) Obscuration by adjacent-internal or superimposed-external radio-opaque or electronic materials. In addition, LLIED categories and/or types might be confused with each other by the interpreting radiologist because of (1) LLIEDs having remarkably similar appearances and positions on a frontal CXR (typically the only view acquired in emergency/trauma department or intensive care unit settings, without a lateral view, revealing LLIED intrathoracic location deep within the right ventricle for an LLP versus subcutaneous within the anterior chest wall for an LLR); (2) General lack of familiarity by a radiologist with LLIED-specific characteristics (especially retained legacy systems or recently introduced devices).^23,28 These fundamental issues are especially germane to the less familiar, infrequently used, much smaller, and more “stringently” MRI-conditional LLIEDs [e.g., pulmonary artery pressure monitor (PAPM) for heart failure^20,24,29] and MRI-“unsafe” LLIEDs [e.g., esophageal reflux capsule (ERC) for pH-monitoring^3,11,12,14], which can easily go unnoticed.

1.3.1.

Artificial intelligence: lead-dependent electronic device recognition on CXR

Other investigators have realized the potential value of CXR-reliant recognition of standard cardiac rhythm-management devices (including lead-dependent pacemakers and cardioverter-defibrillators), for which a comprehensive and detailed manual stepwise visual flowchart CARDIA-X system was initially proposed.³⁰ More recently, an Artificial Intelligence (AI)-based system for CXR identification of lead-dependent devices (which routinely display radiographic text-based identifiers³¹) recognized the device manufacturer and type with 99.6% and 96.4% accuracy, respectively.³² However, the same AI model demonstrated a lower manufacturer-identification accuracy of 71% compared to another AI model running on either a mobile phone application or web platform³³ (accuracy 89% and 73%, respectively), thereby approximating the nonAI-based CARDIA-X performance (i.e., accuracy 85%).³⁴ None of the aforementioned studies or a very recently reported study of only lead-dependent pacemaker detection,³⁵ focused on the recognition of the continuously evolving array of much-smaller modern LLIEDs (which do not display radiographic text-based identifiers).

1.3.2.

AI: opportunity for assisting radiologists in CXR-based LLIED recognition

Thus, AI-based assistance to radiologists in the prompt and confident frontal-CXR detection and localization of any general category of LLIED, and then the identification of its specific type, prior to a scheduled or urgent MRI could have significant safety and operational benefits. In response, our group previously developed a potentially high-performing cascading AI model, described technically elsewhere.³⁶

Unlike the previous basic-technology phase of our research,³⁶ this work focused on the pre-deployment assessment of our combined LLIED-detection and identification AI model for its current readiness, as well as the operational prerequisites to potentially assisting radiologists (reliably, effectively, and efficiently) once truly deployed in real-world clinical practice.^37,38 The evaluations included: (1) Accuracies in the identification of each specific LLIED-type, and consequently the related MRI-safety level, based on experiences during both model development and simulated trialing;^39,40 (2) Clinical MRI-safety implications of observed LLIED nondetections or misidentifications;^39,40 (3) Anticipated (or unanticipated) infrastructure-architectural and/or workflow requirements for productive real-world clinical deployment;^41–43 (4) Expectations and challenges related to ongoing model adaptation to changing real-world conditions.^44–46

2.1.1.

Ground-truth LLIED-type labeling of CXR images

As previously detailed,³⁶ Institutional Review Board-approved retrospective data-mining (spanning: March 1993 to February 2021) allowed the organization-wide extraction of digital CXR examinations (i.e., “AI model development population”) representing a wide range of LLIEDs supporting the development of an AI methodology for device detection followed by identification.³⁶ The specific identities of the LLIED types represented, and their associated clinical implications, were not profiled in the previous nonclinical technical note.³⁶

Serving as project “ground-truth” expert, a fellowship-trained cardiothoracic radiologist with 37 years of experience used a local graphical user interface (GUI)^36,47 to manually delineate the specific LLIED type(s) demonstrated on a CXR image from the AI model development population.³⁶ The frontal view (i.e., Postero–Anterior aka P–A, or Antero–Posterior aka A–P) from each CXR examination was correspondingly labeled using the interactive region-of-interest (ROI) capabilities of the GUI,³⁶ with circular markers applied to derive square ROIs for input into model development (Fig. 1).^36,47

Fig. 1

LLIED-type labeling. Using ROI-annotation capabilities of the GUI, examples of the nine LLIED types represented during the development of the original model are shown on frontal views. The types are arranged in order of decreasing levels of MRI-related risk and/or lessening requirements for patient or LLIED assessment (e.g., cardiology pre-, intra-, and post- MRI evaluation) as follows: (a) Unsafe ERC (Bravo™ Reflux Capsule); (b) Stringently Conditional PAPM (CardioMEMS™ HF); (c) Assessment-requiring conditional LLPs (Nanostim™* and Micra™); (d) Simple conditional LLR (CONFIRM™*); (e) Simple conditional LLRs (Reveal™ XT* and Reveal LINQ™); (f) Simple conditional LLR (CONFIRM Rx™); (g) Simple conditional LLR (BioMonitor2-AF). (* = Legacy LLIED no longer being implanted but possibly retained).

The LLIED categories, including ERC (one type), PAPM (one type), LLP (two types), and LLR (five types), denoted decreasing levels of MRI-related risk and/or lessening requirements for patient or LLIED evaluation (i.e., unsafe, stringently conditional, assessment-requiring conditional, or simple conditional, respectively) (Table 1 and Fig. 2).

Table 1

LLIED categories/types and MRI-safety levels represented.

Fig. 2

LLIED categories/types represented. LLIED-types in AI model development population (Above) and new types appearing in methodology trial population.

During ROI labeling of instances of the original nine LLIED types on CXR images, a basic quality grade reflecting general conspicuity and detail clarity was applied as follows: (1) Unequivocally diagnostic supporting IDentification (“ID” in 76%); (2) Potentially nonrecognizable (“NR” in 12%) for detection or ID; (3) ID with superimposed or abutting materials, or incomplete inclusion within view margins, causing over-lapping (“OL” in 10%); (4) Combined (“NR and OL” in 2%).³⁶

2.1.2.

LLIED-type recognition by original LLIED Model

To optimize data use from the AI model development population during training, validation, and testing of the nine-class “original LLIED model,” conventional approaches to data distribution, expansion, and augmentation (including LLIED-specific inclusion of labeled diagnostic lateral views: Table 2) were employed.³⁶

Table 2

Criteria for LLIED lateral view exclusion from use in model development.

As previously detailed,³⁶ a two-tier system underlying the original LLIED model for LLIED recognition was used: (1) First, to emphasize the detection of device presence and location; (2) Second, to support device-type identification, if detected and then classifiable.³⁶ Ultimately, this prompted the creation of a cascading neural network methodology as follows (Fig. 3).

Fig. 3

Two-tier cascading methodology and data flow for generic LLIED detection and classification³⁶ (GBB, generated bounding box; IoU, intersection-over-union).

2.2.1.

Performance evaluations of original LLIED model for LLIED recognition

2.2.2.

Essential technical developments supporting real-world model deployment and adaptation

A component-based simulation of deployment of our methodology for AI-based LLIED detection and identification on CXR was considered consistent with several recent FDA-endorsed actions.⁴⁴ Hence, we pursued the following opportunities to facilitate the utilization of verified AI model output by the CXR-interpreting radiologist (Table 3; Appendix A).^{40–44,65,66}

Selection/development of viewer for AI model inference-result display and adjudication

Our deployment simulation initially relied on the previously described GUI^36,47 for model inference-result display to the end-user [Table 3]. A zero-footprint (ZF) viewing platform (aka “ZF GUI/viewer”) has since been designed to support all phases of imaging-AI model development and evolution in a user-interactive fashion (Appendix B with Fig. 5).^43,66,67

Fig. 5

Development of viewer for model inference-result display and adjudication. As shown above, the developed ZF GUI/viewer: (1) Has basic functionalities under two main categories [(a) Server-level connectivity; (b) web-based user interface]; (2) Stores information about all image annotations, AI-model inferences, and user responses in an SQL server backend database; (3) Caches all images to be displayed to users in an Orthanc-based DICOM server; (4) Can be invoked from the PACS viewer via URLs passing image-specific parameters (e.g., accession numbers); (5) Can be summoned by clinical users via EMR systems by medical record numbers or accession numbers; (6) Facilitates user interactions, including single-sign-on logins enabled by authorization servers; (7) Supports FHIR interconnectivity (e.g., for placing order messages invoking model inference-result display on specific images); (8) Can absorb traditional HL7 order messages. As represented below, the ZF GUI/viewer is designed to be potentially integrated into the clinical PACS-support infrastructure.

2.2.3.

Performance evaluations of updated LLIED model for LLIED recognition

Performance results from the updated LLIED model were analyzed (Table 3), as previously described, including the following.

2.3.1.

AI technical infrastructure

All AI-model computations utilized several secure on-site graphics processing unit (GPU)-dependent systems. For training, validation, and testing of our AI models, an eight-GPU system [DGX A100 from Nvidia (Santa Clara, California)] was employed.

2.3.2.

Statistical analysis

As part of the standard analysis of testing results related to general LLIED detection in tier 1, precision–recall curves were plotted to reflect the basic comparison between the AI model output and ground-truth expert determinations.^36,72 Tier-2 assessment of the discrimination performance of the multiclass AI model for LLIED-type identification used the area under the receiver operating characteristic curve (AUC ROC) methodology.^36,73

3.1.1.

Cross-validation assessment

As previously reported (without disclosure of LLIED identities),³⁶ tier 1 of the original LLIED model achieved the required 100% LLIED-detection sensitivity during testing.

In this work, during fivefold cross-validation, the mean average precision (mAP) was found to be 0.99 (Fig. 6), indicating the durability of the original LLIED model for LLIED detection and localization.

Fig. 6

Precision–recall curves for two-class detection (tier 1) by original LLIED model.

However, as previously mentioned, meaningful tier 2 cross-validation assessment of identification accuracies was precluded.

3.1.2.

Safety-level and specific-type identification accuracies during model testing

Also as previously described,³⁶ tier 2 of the original LLIED model reached high generic performance levels for LLIED classification. Of those classified as LLIED types, the identification assignments were overall correct at 98.9% during model testing in the AI model development population.³⁶

In this work, AUCs (rounded to nearest 1/100th) for identification of MRI-safety level category (i.e., unsafe, stringently conditional, assessment-requiring conditional, or simple conditional) consistently matched or exceeded 0.98, accompanied by high sensitivities ($\ge 99\%$) and specificities ($\ge 90\%$) (Table 4).

Table 4

Original LLIED model for safety-level identification—model testing.

Identification accuracies for the original nine specific LLIED types were also high with AUC 1.00 for eight types and 0.92 for one LLR type (Table 8).

3.1.3.

Predeployent trialing

Based on postinference ground-truth judgments, the results of the imitated basic real-world trialing experience in the 150 unannotated frontal CXRs from the methodology trial population were strong. They demonstrated the following: (1) maintained detection sensitivity of 100% at the temporary cost of increased GBBs (total 682) from tier-1 processing, with most FP GBBs immediately eliminated transparently by tier-2 processing (i.e., 446 of 682 GBBs excluded) and then the remaining via ground-truth adjudication of inference results (i.e., 135 displayed FP GBBs disqualified by end-user); (2) Ongoing high specific-type identification accuracy at 94.6% (87 of 92 LLIEDs) if preestablished corresponding classes were present at the time of tier-2 processing of the original LLIED model (Fig. 7).

Fig. 7

Basic pre-deployment trialing in methodology trial population. (FP, False positive; GBB, generated bounding box; IoU, intersection-over-union; PPV, positive predictive value).

Of the 101 LLIEDs represented in the methodology trial population, most with corresponding classes within the original LLIED model were correctly identified per safety-level category, with a high overall accuracy of 98% (99 of 100, with sensitivities, $\ge 95\%$ and specificities $\ge 90\%$) (Table 5). Specific LLIED types were also identified with high accuracy (Table 9).

Table 5

Original LLIED model for safety-level identification—basic trialing.

Due to the 100% detection sensitivity achieved by tier 1 of the original LLIED model, no LLIEDs went undetected in the just-described experiences related to either the AI model development population or methodology trial population. However, of the cases misidentified when there were corresponding classes (10/878 = 1.1% of LLIED-related ROIs in AI model development population and 5/101 = 5.0% of LLIED-demonstrating frontal CXRs), the majority [11 of 15 = 73%, representing 8/10 cases and 3/5 cases, respectively (Tables 8 and 9)] could be attributed to suboptimal image-quality grades (cumulatively five NR and OL, four NR, and two OL) (Fig. 8). However, in the methodology trial population, the overall majority of misidentified cases (9 of 14 cases) were ascribed to prior absence of corresponding classes in the original LLIED model for the three new LLR types; this necessitated adjudication correction of the inference results by the ground-truth expert (Fig. 9) for future model modernization including the development of the needed new classes.

Fig. 8

Misidentified LLPs by the original LLIED model due to suboptimal image quality. (a) During model testing (Table 8), an LLP misidentification was attributable to OL image quality with extraordinary superimposition of an LLR (i.e., Reveal LINQ™ identified with 99.9% probability) on the LLP (i.e., Micra™ identified next with 21.6% probability). (b) During basic trialing (Table 9), a misidentified LLP (i.e., Micra™ correctly identified with 4.7% probability, after a reveal LINQ™ identified with 99.9% probability) was attributable to NR/OL image quality related to poor general conspicuity and detail clarity, as well as to superimposed sternal wires.

Fig. 9

ZF GUI/viewer demonstrating inference results (location and probabilities) for end-user adjudication on three new LLIED types in methodology trial population. Previously, unclassified LLR types (i.e., A and B = LUX-Dx™; C = LINQ™ II; D = BioMonitor III) were properly detected as simple conditional LLRs by the original LLIED model, although misidentified as the most common LLR (i.e., Reveal LINQ™) of equal MRI safety (i.e., simple conditional). A previously classified assessment-requiring conditional LLP (i.e., C = Micra™) was both correctly detected and identified at a high probability level, with an appropriate inference GBB label automatically assigned. The user-friendly inference-adjudication capabilities of the ZF GUI/viewer allowed manual label reassignment of inference GBB labels from a drop-down list (e.g., Reveal LINQ™ in A relabeled as LUX-Dx™ in B), or confirmation of correct automatic assignment (e.g., Micra™ in C). In addition, an “other” option for labeling of false-negative or unanticipated future LLIEDs by the end-user is also included. All end-user adjudications of inference results are recorded in the ZF GUI/viewer backend database in support of model continuous learning and modernization.

3.2.1.

Cross-validation assessment

As with the original LLIED model, tier 1 of the updated LLIED model achieved 100% LLIED detection sensitivity during testing. During fivefold cross-validation, the mAP of the updated LLIED model was again 0.99, indicating its detection durability. However, as in the case of the original LLIED model, due to significant data imbalance (with some no-longer-implanted legacy LLIED types or new LLIED types still represented by very small patient subsets) meaningful cross-validation assessment of identification accuracies (safety-level or specific-type) could not be adequately evaluated.

3.2.2.

Safety-level and specific-type identification accuracies during updated model testing

Like with the nine-class original LLIED model in the AI model development population, tier-1 LLIED-detection of 100% was followed by high classification performance for LLIED identification by the 12-class updated LLIED model in the AI model update population, with the identification assignments overall correct at 99.5% during model testing.

AUCs for the identification of the category of MRI-safety level (i.e., unsafe, stringently conditional, assessment-requiring conditional, or simple conditional) consistently matched or exceeded 0.99, accompanied by high sensitivities ($\ge 99\%$) and specificities ($\ge 90\%$) [Table 6].

Table 6

Updated LLIED model for safety-level identification—model testing.

For the identification of the original 9, plus three new, specific LLIED-types, AUCs were 1.00 for nine types, and 0.92 to 0.99 for three LLR types (Table 10).

Of the five misidentified LLR cases, the updated LLIED model displayed on the ZF GUI/viewer the correct label assignment as the second, third, and fourth most likely in 2, 2, and 1 case(s), respectively. Suboptimal image quality was applied to two (both NR) of the five misidentified cases.

3.2.3.

Limited-deployment simulated real-world trialing of updated LLIED model

The initial use of the ZF GUI/viewer in our near-real world clinical test environment, with its DICOM-SR output for this project, supports immediate model inference-result presentation (including 0% to 100.0% probability display) simultaneously with the CXR examination posting on the clinical PACS worklist. The previously described purposeful display-limitation of stacked overlapping and identically labeled inference-GBBs to the one GBB with the highest probability level on a case-by-case basis enhanced end-user experience by eliminating an extra 1 to 17 noncontributing overlapping identically labeled GBBs in 63 of the 100 cases. The result was a remarkably simpler inference-result adjudication process without loss of model performance.

When combined, these capabilities facilitated user-friendly adjudication of inference results (by conventional clicking) within seconds, including (1) Acceptance of a result correctly identifying an LLIED; (2) Correction (relabeling) of a misidentified LLIED result; (3) Result rejection by simple passive disregarding of a false-positive nonLLIED GBB (Fig. 10).

Fig. 10

Simultaneous adjacent clinical PACS and ZF GUI/viewer displays. In the limited-deployment test environment, the standard clinical PACS display (a) is simultaneously accompanied by an adjacent display of inference results by the ZF GUI/viewer (b) on the same monitor used by radiologist for routine clinical image interpretations, although not yet integrated. On this frontal CXR image, the 12-class updated LLIED model has correctly detected/localized and identified (with GBB) the MRI-unsafe ERC (Bravo™ Reflux Capsule), with immediate single-click positive adjudication (Red boundaries applied with acceptance) versus passive rejection of any false-positive GBBs (maintained blue boundaries).

The results of the prospective application of the 12-class updated LLIED model within the parallel ZF GUI/viewer test environment in the 100-case updated methodology trial population were also strong. Following tier-1 100% detection of all 101 LLIEDs (two LLIEDs in one case), strong tier-2 overall accuracy of 97% (98 of 101; sensitivities 75% to 98% and specificities 92% to 93%) for the identification of safety-level category was achieved; stringently conditional and unsafe categories were not represented in this experience (Table 7).

Table 7

Updated LLIED model for safety-level identification—limited deployment.

Specific LLIED types were also identified with a strong overall accuracy of 95% (96 of 101 LLIEDs, including all but 1 of the 14 examples of the three newly classified LLR types) (Table 11).

Of the five misidentified LLIED cases, a valid GBB displaying the correct LLIED-type label was shown by the ZF GUI/viewer as being the second most likely in two cases and third most likely in three cases. Suboptimal image quality was noted in three (two OL and one NR) of the five misidentified cases.

4.1.1.

Uniqueness of LLIED use-case and developed AI model

The practical clinical use-case^65,66 inspiring our initial development³⁶ is distinctively different from the most closely corresponding pursuits,^30–35 due to its focus on the continuously evolving array of modern much-smaller LLIEDs being inserted into the chest with greater frequency. To our knowledge, this is the first reported achievement of AI-based radiographic detection and identification (important to FDA recalls, such as the Nanostim LLP for dysfunction, as well as to MRI safety) directed at LLIEDs, ranging from MRI-conditional to MRI-unsafe.

From the beginning, this work emphasized real-world conditions^{36–38,40,41,43–46} by (1) Utilization of large datasets representing multiple geographically dispersed sites for model development; (2) Representation of varying general radiographic technology producing digital CXRs over almost three decades; (3) Inclusion of all LLIED image qualities (e.g., NR, OL, and NR and OL, cumulatively representing 24% and 30% of AI model development population and methodology trial population, respectively); (4) Model retraining to account for previously unclassified LLIED types (i.e., creating a newer 12-class updated LLIED model to replace the original nine-class model); (5) Simulation of initial real-world trialing of both LLIED AI models on separate patient series (i.e., methodology trial population and updated methodology trial population).

4.1.2.

LLIED detection/localization and identification performance of the adapting AI model

We found both the 9-class and 12-class LLIED AI models to consistently achieve the premandated 100% detection/location sensitivity (in tier 1) in all described pre-deployment experiences; the durability of the two models was confirmed by fivefold cross-validations. In addition, both models consistently achieved high identification accuracies (in tier 2) for MRI-safety category and specific-type in all reported evaluations, including mimicked real-world trialing (i.e., 98% and 97% correct safety-level categorizations in the methodology trial population and the updated methodology trial population, respectively).

4.1.3.

Clinical implications of the adapting LLIED model

Due to the strength of tier-1 processing in our cascading AI methodology, no LLIEDs went undetected in any of the described experiences with either the original LLIED model or the updated LLIED model. Tier 2 related misidentifications were uncommon and most often attributable to suboptimal image quality.

When misidentifications were considered from an MRI-safety standpoint, it was noted that in our reported cumulative pre-deployment experience, there were no cases of tier-1 nondetection and/or tier-2 misidentification of either an MRI-stringently conditional PAPM (i.e., CardioMEMS™ HF) or an MRI-unsafe ERC (i.e., Bravo™ Reflux Capsule), even when an LLIED type was not previously classified. Thus, end-user adjudication of displayed inference results on these two more risky LLIED categories/types was consistently positive, thereby fully supporting higher levels of awareness of greater potential MRI risk in affected patients.

In the presence of corresponding classes for tier-2 processing, only 21 instances of MRI-conditional LLIED misidentification were found in the following decreasing order: (1) Simple Conditional LLR misidentified as another LLR (eight instances); (2) Simple Conditional LLR over-identified as an assessment-requiring conditional LLP (i.e., Micra™) (seven instances); (3) Simple conditional LLR over-identified as a stringently conditional PAPM (i.e., CardioMEMS™ HF) (three instances); (4) Assessment-requiring conditional LLP (i.e., Micra™) under-identified preadjudication as a simple conditional LLR (three instances). Respectively, the related potential clinical safety and operational implications included (1) No negative impact; (2) Premature operational considerations (e.g., unnecessary engagement of cardiology for peri-MRI assessments); (3) Premature safety considerations (e.g., plans to over-emphasize more basic forms of scanning); (4) Initial underestimation of needed coordination of operational support (e.g., failure to engage cardiology for needed peri-MRI assessments). However, it is important to realize that, as a decision-support assistant, the inference results generated by our LLIED methodology (with 100% LLIED detection/localization) are displayed directly to the radiologist for their adjudication before clinical use. Therefore, such inconsistencies are likely temporary and become corrected during the regular workflow, which is designed to actively involve the radiologist (rather than to function autonomously) and, hopefully in the future, is enhanced through integration with the EMR.