Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

CBCT-to-CT synthesis with a single neural network for head-and-neck, lung and breast cancer adaptive radiotherapy

CBCT-to-CT synthesis with a single neural network for head-and-neck, lung and breast cancer... Purpose: CBCT-based adaptive radiotherapy requires daily images for accurate dose calculations. This study investigates the feasibility of applying a single convolutional network to facilitate CBCT-to-CT synthesis for head-and-neck, lung and breast cancer patients. Methods: Ninety-nine patients diagnosed with head-and-neck, lung or breast cancer undergoing radiotherapy with CBCT-based position veri cation were included in this study. CBCTs were registered to planning CTs according to clinical procedures. Three cycle-consistent generative adversarial networks (cycle-GANs) were trained in an un- paired manner on 15 patients per anatomical site generating synthetic-CTs (sCTs). Another network was trained with all the anatomical sites together. Performances of all four networks were compared and evaluated for image similarity against rescan CT (rCT). Clinical plans were recalculated on CT and sCT and analysed through voxel- based dose di erences and -analysis. Results: A sCT was generated in 10 seconds. Image similarity was comparable be- tween models trained on di erent anatomical sites and a single model for all sites. Mean dose di erences < 0:5% were obtained in high-dose regions. Mean gamma (2%,2mm) pass-rates > 95% were achieved for all sites. Conclusions: Cycle-GAN reduced CBCT artefacts and increased HU similarity to CT, enabling sCT-based dose calculations. The speed of the network can facilitate on- line adaptive radiotherapy using a single network for head-and-neck, lung and breast cancer patients. arXiv:1912.11136v1 [physics.med-ph] 23 Dec 2019 Contents I. Introduction 1 II. Material and methods 2 II.A. Imaging protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 II.B. CBCT-to-CT synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 II.B.1. Image pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 II.B.2. Network architecture and training . . . . . . . . . . . . . . . . . . . . 4 II.B.3. Image post-processing . . . . . . . . . . . . . . . . . . . . . . . . . . 6 II.C. Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 II.C.1. Image comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 II.C.2. Dose comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 III. Results 8 III.A. Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 III.B. Image comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 III.C. Dose comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 IV. Discussion 15 Acknowledgements 17 Con ict of Interest Statement 17 References 17 ii CBCT-to-CT synthesis with a single neural network page 1 I. Introduction In modern external beam image-guided radiotherapy (IGRT), cone-beam computed tomo- 1,2,3 graphy (CBCT) plays a crucial role in accurate patient position veri cation . Also, CBCT 4,5 can facilitate adaptive radiotherapy (ART) by visualising daily anatomical variations . CBCT image quality is inferior to that of CT in soft-tissue contrast and Houns eld Units 6,7,8 (HU) consistency due to the presence of artefacts . Therefore, CBCT is not sucient to perform accurate dose calculations and patients need to be referred for a rescan CT (rCT) whenever anatomical di erences are noted between daily images and planning CT . However, scheduling and acquiring a rCT adds logistic complexity and patient burden to the treatment. On the contrary, with ART these issues can be addressed by exploiting the daily CBCT images to reduce set-up errors and eliminate the need for an rCT . A prerequisite for online ART is that the CBCT quality and HU accuracy is sucient to enable dose calculation. Considerable literature has recently emerged proposing to correct CBCT imaging arte- 11,12 facts and increase image intensity consistency using: look-up table-based approaches , deformable imaging registration (DIR) of the planning CT to the daily anatomy on 13,14,15 CBCT and model- or Monte Carlo-based methods for scatter estimation and correc- 16,17,18 19 tion . Speci cally, DIR enabled accurate dose calculations for head-and-neck (HN) but obtained lower dose accuracy in more complex anatomical changes such as lung and 20,21 20,22,23 pelvis . Also, Monte Carlo-based methods were suitable for ART . These tech- niques can be deployed on a time scale of minutes, which is not acceptable when aiming to use CBCT images for daily online dose evaluation or online pre-treatment adaptation. Recently, deep learning has been proposed for fast CBCT artefact correc- 24,25,26,27,28,29 tion . Deep learning is a branch of arti cial intelligence and machine learning that involves the use of neural networks to generate a hierarchical representation of the in- 30,31 put data to achieve a speci c task without the need of hand-engineered features . Deep learning has shown promising results solving image-to-image translation problems within sec- 32,33 onds . In this sense, previous work demonstrated the use of a two-dimensional (2D) U-net 24,25,26 to improve CBCT image quality . Moreover, it has been shown that converting CBCT 27,29 with deep learning resulted in accurate dose calculation for prostate cancer patients and Last edited: December 25, 2019 page 2 Maspero M 28,34 HN cancer patients . In this study, we investigate whether CBCTs converted with convolutional networks may be used as a surrogate of the daily anatomy for dose calculations. We employ a network trained in an unpaired manner to convert CBCT-to-CT of HN, lung, or breast cancer patients investigating whether a single network can generalise for the three anatomical sites. A single network trained for all the anatomical sites was compared to three networks trained per anatomical site. Performances of all four networks were compared and evaluated for image similarity and dose calculation accuracy between CT and rCT. II. Material and methods II.A. Imaging protocols Ninety-nine patients diagnosed with HN (33), lung (33) or breast (33) cancer undergoing radiotherapy were retrospectively included in this study. Irradiations were performed be- tween May 2016 and February 2019 on Agility linacs (Elekta AB, Sweden) with CBCT-based pre-treatment position veri cation. An rCT was acquired in case anatomical variations were noted on the CBCT. We included at least fourteen patients with rCT per site. The (r)CTs were acquired on a Brilliance Big Bore (Philips Healthcare, Ohio, USA); CBCTs were acquired using X-ray volumetric imaging (XVI, v5.0.2b72 Elekta AB, Sweden) system. Table 1 reports the imaging protocols for CT, rescan CT (rCT) and CBCT for all the patients included in the study. CBCTs were acquired with 0.25 rotation/s gantry speed and 5.5 frames/s. All the CBCTs were acquired with a 200 -arc utilising an empty lter cassette (F0) in combination with a centred detector panel (S position, maximum FOV=27x27 cm ). The eld-of-view (FOV) was in four cases (elective lymph-nodes irradiations or double-sided irradiation for breast and HN patients) enlarged to a maximum of 41x41 cm using a shifted detector panel (M position) to accommodate the CTV in the CBCT FOV. Imaging frequency of CBCT followed the extended non-action limit protocol : online corrections (action level 0 mm) were applied in the case of partial or ablative breast irradia- tion, and oine long (N=3, P=5) and short (N=2, P=3) scheme were applied for irradiations II. MATERIAL AND METHODS CBCT-to-CT synthesis with a single neural network page 3 Table 1: Overview of CT (including also rescan (r)CT) and CBCT imaging protocols in terms of eld-of-view (FOV), acquisition matrix (Acq matrix), resolution (Res), tube voltage (kVp), exposure (ms) and current (mA). For exposure and current, the mean value () was reported along with the range. a a b b b FOV Res Voltage Exposure Current Modality Site Acq matrix 3 3 [cm ] [mm ] [kVp] [ms] [mA] 43-70 512 0.83-1.37 98365 15950 Head-and-neck 43-70 512 0.83-1.37 120 923-1090 47-271 30-111 101-535 2-3 47-70 512 0.92-1.37 1050109 6337 (r)CT Breast 47-70 512 0.92-1.37 120 923-1332 31-271 31-120 103-400 2-3 29-70 512 0.57-1.37 38863095 9866 Lung 29-70 512 0.57-1.37 120 500-10091 30-271 23-220 76-660 1-3 27 135-270 1-2 115 143 Head-and-neck 27 135-270 1-2 100 10-40 10-20 13-53 126-526 1-2 27-41 270-540 0.5-1 332 172 CBCT Breast 27-41 270-540 0.5-1 120 32-40 16-20 26-53 262-526 0.5-1 27 270 1-2 316 201 Lung 27 270 1-2 120 10-40 16-25 26-53 128-528 1-2 Expressed in RL, AP, FH directions; the range is reported in terms of min-max. Reported in terms of mean value and range=min-max. Except for H18 and H20 where kVP was 120. Except for L11, L22 and L24 where kVP was 100. having > 20 and < 20 fractions, respectively. Imaging frequency may have been increased after consultation between a medical physicist and a radiotherapist on a single patient-basis in case large inter-fraction motions were observed in the initial fractions or whenever RT technicians reported diculties in reproducing the planning position. CBCTs were translated to apply clinical set-up corrections and resampled to the plan- ning CT within the X-ray volumetric imaging (XVI, v5.0.2b72 Elekta AB, Sweden) system. Registrations were estimated within a clip-box including the CTV based on bone rigid (trans- lation and rotation) matching . For the breast patients treated with local RT followed by a sequential boost, a dual rigid registration was performed based rst on bone matching 37,38 followed by grey level (soft-tissue) matching . The centre of rotation was assigned as the centre of the PTV. Last edited: December 25, 2019 II.A. Imaging protocols page 4 Maspero M II.B. CBCT-to-CT synthesis II.B.1. Image pre-processing Before supplying images to the network, CT and CBCT were cropped to the size of the CBCT FOV after identifying the so-called \Mask " according to the following steps. CBCTs were CBCT thresholded at -999.9, obtaining a binary mask. In each transverse slice containing the binary mask, morphological closure was performed, and the smallest bounding box containing the mask was found. The biggest circle contained in the bounding box was searched starting from a radius of 26.9 cm and iteratively increasing its size. The circle was propagated for all the slices obtaining Mask . CT and CBCT were cropped in the bounding box containing CBCT Mask . CBCT In addition to cropping, voxel intensity of CT and CBCT were clipped within the interval [1000;3071] HU and image intensity was linearly rescaled to 16-bit. II.B.2. Network architecture and training To generate CT from CBCT, a 2D cycle-generative adversarial network (cycle-GAN) was adopted . Cycle-GANs enable unpaired training, which, compared to paired training, makes the network less sensitive to residual mismatch of CT and CBCT . The network consisted of two cycles called \forward" and \backwards" during which GANs generated CT from CBCT and vice-versa. Moreover, so-called \cycle-consistency" was enforced with an L -norm such that after converting from CBCT to CT and vice-versa, the original image should be obtained. The architecture, based on the cycle-GAN provided by 39 41 Zhou et al. , was implemented in Tensor ow (v1.3.0). Nine-blocks residual networks were employed as generators and Patch-GANs as discriminators (Figure 1). Stochastic gradient descendent was used applying an Adam solver with learning rate = 0.0002, momentum parameters = 0.5 and = 0.999. Instance normalisation was employed with a batch 1 2 size of 1. The weights of the network were randomly initialised from N (0; 0:02). Weight op- timisation was performed as in Goodfellow et al. alternating between one gradient descen- dent step on the discriminator network and one step on the generator network after having performed a forward and backward cycle. A structured loss function GAN+  L +cycle- consistency with  = 25 was adopted. The original implementation of the network by Zhou II. MATERIAL AND METHODS II.B. CBCT-to-CT synthesis CBCT-to-CT synthesis with a single neural network page 5 Figure 1: Architecture of the nine-block residual network used as a generator (top) and of the convolutional network called Patch-GAN used as a discriminator. The size of the images is numerically reported, except for the residual block, where it remains stable. Note that the nine-blocks are omitted in the schematic. Each of the lters had stride two, kernel size four; leaky rectilinear recti er unit had a scalar multiplier of 0.2, padding was applied in re ect mode. et al was modi ed to accommodate 16-bit grey-scale images with a size of 256x256. To train, validate and test the network, patients were split into three datasets: 15 patient per site for training, 8 for validation and the remaining 10 for test. The validation set was used to aid hyperparameter optimisation and to determine at which iteration the https://github.com/xhujoy/CycleGAN-tensorflow Last edited: December 25, 2019 II.B. CBCT-to-CT synthesis page 6 Maspero M training could be stopped to avoid over- tting (early-stopping), while the test set was used to evaluate the performance of the network. To investigate the impact on dose calculation of a di erent CT, the patients included in the test set were selected among the patients with an rCT and with CBCT and rCT acquired with minimal time di erences. Patients' demographics were controlled to ensure data balancing in terms of the number of patients in the three sets. Also, we inspected the ratio of male/female, distribution age, tumour staging distribution and linac on which CBCT were acquired (Supplementary Material). Training of the cycle-GAN was performed in the transverse plane, and for each iteration, random CT, and CBCT slices of di erent patients were supplied. Three networks were trained separately on each anatomical site. Another network was trained on all anatomical sites to investigate whether a single network may generalise for all anatomical sites. The networks were trained for 200 epochs on a Tesla P100 (16 Gb, NVIDIA, California, USA) graphical processing unit (GPU) with batch size one and image pool of 1000 images. Data augmentation was applied during training by ipping the images left and right and randomly cropping of 30x30 voxels after having bi-linearly resampled the images to 286x286 voxels in Mask . Early stopping was applied selecting the earliest epoch for which average L in CBCT 1 the body contour over the pts of the validation test was the lowest and within one  from L calculated at every 10 epoch (a total of 20 models were stored, one each 10 epochs). The total amount of slices utilised during training was reported to verify data balancing among sites: 3668 transverse slices acquired in a minimum of four di erent linacs were used for training, composed by 1606, 1046 and 1016 slices from HN, lung and breast cancer patients, respectively. II.B.3. Image post-processing First, the trained model was applied within Mask to the pre-processed CBCTs (as de- CBCT scribed in II.B.1.) obtaining 16-bit images. Then, the HU intensity range of [-1000;3071] was restored with a linear rescaling obtaining the so-called CBCT . CBCT were conv conv bi-linearly resampled from a matrix size of 256x256 to the original CBCT resolution. To generate images for the full CT FOV, the CBCT was substituted in the original conv CT within the Mask . The image obtained combining CT and CBCT will be referred CBCT conv to as synthetic-CT (sCT) (Figure 2). II. MATERIAL AND METHODS II.B. CBCT-to-CT synthesis CBCT-to-CT synthesis with a single neural network page 7 Figure 2: Schematic of the image work ow for a 2D transverse slice of a breast cancer patient. After image acquisition, registration (1) and pre-processing (2) the trained network is deployed producing converted CBCT (CBCT , 3) which substituted the original CT within Mask conv CBCT obtaining the so-called synthetic CT (sCT). II.C. Evaluation Image evaluation in terms of similarity between sCT and rCT was performed to assess whether the single network trained with all the anatomical sites was comparable to the three networks trained per anatomical site. If performances were comparable, the single network was considered to assess the appropriateness of the CBCT conversion with the trained cycle-GAN on the test set with an image and a dose comparison. II.C.1. Image comparison Similarities between the image intensity of sCT, CBCT, CT and rCT were calculated within Mask in terms of mean absolute error (MAE) and mean error (ME) as proposed by Liang CBCT et al. . Rescan CT was considered as ground truth, and the metrics were calculated in terms of mean 1 and range. Wilcoxon signed-rank tests were conducted between sCT/rCT and CT/rCT for MAE. Additional metrics are reported in Supplementary Material. Last edited: December 25, 2019 II.C. Evaluation page 8 Maspero M II.C.2. Dose comparison For the patients in the test sets, clinical plans were recalculated on CT, sCT and rCT images in Monaco (v 5.11.02, Elekta AB, Sweden) using a Monte Carlo algorithm on a grid of 3 mm with 5% and 3% statistical uncertainty for volumetric modulated arc therapy (VMAT) and intensity-modulated radiotherapy (IMRT) plans, respectively. Clinical contours, delineated by a radiation oncologist on the planning CT, were rigidly transferred to the sCT and rCT except for the body contour, which was automatically re-delineated. These contours were considered as volumes of interest (VOIs). sCTrCT Dose distributions were analysed through relative dose di erences (DD = sCT rCT CTrCT and DD = ) in the high dose region (dose > 90% of the prescribed dose). Also, CT rCT 3D -analysis with 3%,3mm and 2%,2mm criteria relative to dose on rCT within regions of 50% prescription dose were performed. For all dose comparisons, a 15 mm cropping in the proximity of body contour was performed to take account of dose build-up in the proximity of the skin . To investigate the impact of dose di erence within VOIs, analysis of dose-volume his- togram (DVH) points was performed on sCT and rCT. The DVH points analysed were the maximum dose and mean dose. OARs were considered for such analysis when they were present in at least four of the patients for each anatomical site: submandibular and parotid glands, spinal cord, larynx and brain stem for head-and-neck patients; lungs, heart, oesoph- agus, humerus and spinal cord for breast patients; lungs, heart, oesophagus, spinal cord and trachea for lung patients. III. Results III.A. Network Cycle-GANs required about eight days and ve hours training on a GPU Tesla P100 (NVIDIA Corporation) employing 200 epochs with 3668 slices. As a result of the early stopping investigation, after inspecting the L loss function on the training and validation set, we opted for utilising 160000 (100 epochs), 180000 (160 epochs), 180000 (170 epochs) iterations for the network trained on HN, breast and lung dataset, respectively, and III. RESULTS CBCT-to-CT synthesis with a single neural network page 9 360000 iterations (100 epochs) for the network trained on all the three sites combined. Generating sCT required <10 s for an entire CBCT volume (70 slices) on GPU and about 40 s on CPU. III.B. Image comparison The time between rCT and CBCT in the test set was on average ( [min; max]) 1  3 [0;8] days and 29 11 [8;67] days between CT and CBCT. The increased time between CT and CBCT may result in larger di erences when comparing rCT vs CT compared to sCT vs rCT. Similarity metrics over the test patients are reported in Table 2. Table 2: Overview of the image comparison. Image comparison calculated as mean (1) and range ([min;max]) of the test dataset (30 patients) compared to the reference dataset in terms of mean absolute error (MAE) and mean error (ME) between the Test image minus the Ref image. Site Head-and-Neck Breast Lung MAE ME MAE ME MAE ME Test Ref [HU] [HU] [HU] [HU] [HU] [HU] 19520 -12233 15240 7137 21944 15348 CBCT rCT [160;230] [-183;-71] [98;213] [7;115] [133;280] [94;230] sCT single 5312 -37 6618 -613 8310 -211 rCT network [37;77] [-15;10] [41;95] [-24;13] [72;104] [-25;10] sCT separate 5112 -66 6718 -511 869 -514 rCT networks [35;74] [-16;4] [41;98] [-18;14] [73;105] [-28;10] 6317 -1815 6324 820 9423 922 CT rCT [-40;90] [-46;3] [40;115] [-14;54] [68;146] [-33;36] sCT obtained from a single network trained on all the anatomical sites. sCT obtained from three di erent networks trained on each anatomical site. Generic network vs site-speci c networks No statistically signi cant di erences (p>0.35) were found between networks trained per separate anatomical site and the single network trained with all three anatomical sites. This justi es the use of the single model trained for all the anatomical sites for assessing the accuracy of HU and for the dose comparison. Accuracy of HU One can notice that similarity increased between sCT and rCT compared to CBCT and rCT; e.g. MAE decreased from 19520 (CBCT/rCT) to 5312 HU (sCT/rCT) for HN. All the similarity metrics calculated between sCT/rCT and CT/rCT can be considered equivalent Last edited: December 25, 2019 III.B. Image comparison page 10 Maspero M to the metrics calculated between CT and rCT, with no signi cant di erences (p>0.14) for all the three anatomical sites. The mean MAE and range for sCT/rCT were smaller than for CT/rCT due to the reduced time between sCT/rCT, which resulted in less anatomical di erences. Figure 3, 4, 5 show examples of CBCT and sCT obtained from the single network for a HN, breast and lung cancer patient, respectively. One can observe that the network reduced scatter artefacts while retaining anatomical accuracy. Considering an example of lung patient (Figure 5), one can observe the occurrence of atelectasis between CT and rCT/sCT. III.C. Dose comparison Figure 3, 4, 5 report also dose distributions calculated on CT, rCT and sCT along with their DVHs. One can qualitatively notice small di erence between doses on rCT and sCT. When considering the quantitative results, no signi cant di erences were observed between DD and DD . On average (Table 3), dose di erences between sCT/rCT (DD ) sCT CT sCT were lower then for CT/rCT (DD ), e.g. in the high dose region (D> 90%) maximum rCT mean di erences in the range [0.1;0.2]% and [-0.3;0.9]% were found for DD and DD , sCT rCT respectively. Table 3: Statistics of the dose comparison of the thirty patients in the test set. The values are reported as percentage mean1 and range [min; max]. sCT vs rCT CT vs rCT 1 2 3 4 3 4 DD DD 3%;3 mm 2%;2 mm 3%;3 mm 2%;2 mm sCT rCT Sites [%] [%] [%] [%] [%] [%] 0:1 0:5 98:4 1:9 95:3 2:8 0:9 1:1 97:0 2:3 91:1 4:9 Head-and-neck [0:8; 0:7] [93:4; 100] [90:1; 99:6] [1:8; 2:0] [92:7; 99:6] [80:7; 97:4] 0:1 0:4 96:5 4:1 89:4 8:2 0:3 0:8 95:7 5:5 88:8 9:2 Breast [0:5; 0:8] [86:1; 99:6] [70:8; 97:8] [1:7; 0:7] [81:1; 99:5] [66:3; 97:5] 0:2 0:9 97:0 2:9 91:3 6:2 0:1 1:5 95:2 3:6 87:7 6:8 Lung [1:3; 1:8] [91:5; 99:7] [82:3; 98:4] [2:6; 3:0] [89:8; 99:4] [79:4; 96:6] c sCTrCT DD =  100 on dose > 90% of the prescribed dose. sCT rCT rCTCT DD =  100 on dose >90% of the prescribed dose. rCT CT Pass rates of on dose > 50% of the prescribed dose. 3%;3 mm Pass rates of on dose > 50% of the prescribed dose. 2%;2 mm The mean gamma pass rates with the 2%,2mm criteria were higher for sCT/rCT com- pared to CT/rCT for all VOIs, which is in line with the dose di erences observed. All DVH III. RESULTS III.C. Dose comparison CBCT-to-CT synthesis with a single neural network page 11 points di ered on average <0.5% compared to rCT. DVH points di erences were < 2% ex- cept for the heart of a breast patient (B31, -5.6%), the oesophagus of two breast patients (B30, 3.1% and B31, 2.3%), left lung and spinal cord of two lung patients (L25, -2.1% and L27, 3.8%, respectively). Images of the patients with doses di erences in VOIs > 2% were inspected on a single-case basis, as reported in the Supplementary Material. We noticed that large dose di erences were in low-dose regions, which are more sensitive to statistical di erences due to the low amount of events in the Monte Carlo dose calculations. For a lung case (L25), anatomical di erences were reported as the cause of the observed di erences. Also, residual artefacts characterised by inhomogeneous HUs seem to be present along the craniocaudal direction in the lungs for sCT; it appears that for this case the CBCT artefacts were not fully recovered by the network within the lungs. Also for the other lung case (L27), anatomical di erences were observed in the lung. In addition, we noticed the patient was obese and the CBCTs were characterised by severe scatter artefacts. On sCT, the spinal cord was not entirely recovered, possibly resulting in local di erence. Besides, the spinal cord is located in a low-dose region, which may be highlighted when considering metrics as voxel-wise relative di erences. Last edited: December 25, 2019 III.C. Dose comparison page 12 Maspero M H24 sCT , rCT ; Presc dose = 70 Gy 20161019 20161020 CTVp5425 CTVnR5425 SpinalCord ParotidR ParotidL SubmandR SubmandL Larynx Brainstem body 0 20 40 60 80 100 Dose [% to Presc dose] st Figure 3: Sagittal views for the head-and-neck cancer patient H24 of: (1 row) CBCT st nd rd th (1 column), CT (2 column), rescan CT (rCT, 3 column) and synthetic CT (sCT, 4 nd rd column), along with (2 row) the respective di erence to rCT, the doses (3 row). The red, black, or green dotted rectangles indicate the position of Mask . The days refer to CBCT th the acquisition date of the rCT. In the 4 row, the DVH is shown for target and OARs of sCT (continuous lines) and rCT (dashed lines). III. RESULTS III.C. Dose comparison Volume [%] CBCT-to-CT synthesis with a single neural network page 13 B27 sCT , rCT ; Presc dose = 61.18 Gy 20171024 20171025 CTV4669 CTVb6118 CTVnL14669 80 LongL LongR TotalLung Esophagus Heart SpinalCord PRVhumerus 40 body 0 50 100 150 Dose [% to Presc] st st Figure 4: Coronal views for the breast cancer patient B27 of: (1 row) CBCT (1 column), nd rd th CT (2 column), rescan CT (rCT, 3 column) and synthetic CT (sCT, 4 column), along nd rd with (2 row) the respective di erence to rCT, the doses (3 row). The red, black, or green dotted rectangles indicate the position of Mask . The days refer to the acquisition date of CBCT th the rCT. In the 4 rows, the DVH is shown for target and OARs of sCT (continuous lines) and rCT (dashed lines). Last edited: December 25, 2019 III.C. Dose comparison Volume [%] page 14 Maspero M L26 sCT , rCT ; Presc dose = 30 Gy 20170926 20170926 ITV3000 Longptv LongL LongR Esophagus Heart SpinalCord05 Trachea body 0 50 100 150 Dose [% to Presc] st st Figure 5: Axial views for the lung cancer patient L26 of: (1 row) CBCT (1 column), CT nd rd th (2 column), rescan CT (rCT, 3 column) and synthetic CT (sCT, 4 column), along with nd rd (2 row) the respective di erence to rCT, the doses (3 row). The red, black, or green dotted rectangles indicate the position of Mask . The days refer to the acquisition date of CBCT th the rCT. In the 4 row, the DVH is shown for target and OARs of sCT (continuous lines) and rCT (dashed lines). III. RESULTS III.C. Dose comparison Volume [%] CBCT-to-CT synthesis with a single neural network page 15 IV. Discussion Cycle-consistent generative adversarial network (cycle-GAN) increased the accuracy of HU in CBCT, enabling sCT-based calculations for HN, lung and breast cancer patients. Also, we found that a single network trained on all the three sites performed similarly to three networks trained on each anatomical site. When investigating the accuracy of HU on sCT calculating image similarity to rescan CT, we found that HU values were comparable to values observed between CT and rCT. We observed a slight increase in performance for HN compared to breast and lung cancer patients. The network was trained with higher amount of slices for HN (1606) compared to lung and breast (1046 and 1016, respectively). We hypothesise that this data imbalancing may have resulted in relatively increased perfomances for HN cancer patients. Also, the use of immobilisation masks for HN case may increase the reproducibility of patient set- up or reduce motion artefacts in the images (both CT and CBCT) . Though variations in the CBCT imaging protocol were reported, e.g. kV, mAs and linac where the images were acquired, we did not observe any e ect on the quality of sCT. It may be of interest to investigate thoroughly the in uence to the robustness of the method to variations of acquisition settings, as already proposed by Maier et al. . In terms of dose calculation accuracy, we compared sCT to rCT, achieving excellent results for all the anatomical sites. Previous work with deep learning was performed on 24,27,29,49 28,50 25 prostate , HN and lung patients. For HN patients, similar ndings were re- ported by Liang et al. , where also a cycle-GAN was utilised where a mean (1) 2%, 2mm pass-rates of 98.41.7% was obtained compared to 98.41:9% of this work. Also, Li et al. used a 2D U-net with residual convolutional units achieving mean DVH point di erence < 1% . In our study, similar mean DVH point di erences (< 0:5%) were achieved, which demonstrates the high sCT quality achieved with our approach. For lung patients, Xie et al. applied patch-based residual learning on lung patients obtaining a conspicuous correction of cupping and streaking artefacts . Unfortunately, they did not perform any dose calculations and used di erent metrics, making it dicult to compare the studies. Repositioning inevitably occurred between CBCT, rCT and CT. To further minimise anatomical and set-up di erences, we could have recurred to deformable image registration Last edited: December 25, 2019 page 16 Maspero M (DIR) to increase the similarity of CBCT/sCT and CT/rCT. However, we opted against it for the following reasons: (i) since we were trying to reproduce the dose derived by CT- based calculations, we did not want to modify CT or rCT further; (ii) residual deformation errors should be thoroughly evaluated , and this was deemed out of the scope of this investigation; (iii) recurring to using solely translation mimics the set-up procedure that is currently performed clinically at the linacs, and we aimed at observing the impact of dose evaluation in a comparable setting. The main limitation of this study is deemed to be the cohort size: ten patients per anato- mical sites in the test set may be considered as a low number. Before clinical implementation, a study including a larger number of patients should be initiated, paying particular attention to the data variability and data balancing among anatomical sites. Besides, we did not adapt the contours of targets and OARs, which is necessary to investigate the clinical impact of replanning thoroughly. To our knowledge, this is the most extensive study so far presented utilising a convolutional neural network for sCT generation with ninety-nine included pa- tients. Also, notwithstanding the relatively limited sample, this work o ers valuable insights into the generalisation capability of a single cycle-GAN, and, in general, in showing that a single neural network can convert CBCTs of multiple sites. Currently, we balanced the sites based on the number of patients performing training with about 1.5 times more images for HN compare to lung and breast patients. It would be interesting to investigate in a future study whether di erent balancing may maintain com- parable image similarity for all the anatomical sites. We believe that further improvements can be made by balancing data in terms of the number of slices included in the training. In our study, we showed, for the rst time, that a single cycle-GAN can be utilised for multiple anatomical sites as HN, breast and lung. This nding has important implications for simplifying the training of a convolutional network since a single network may be adopted for di erent anatomical sites. To fully understand whether a single network may facilitate CBCT-based dose calculations for the whole body, we are currently performing a novel study including additional anatomical areas, e.g. pelvis, lower abdomen and brain. The impact of our work is that with a single cycle-GAN CBCTs were converted into CTs, resulting in sCTs that have sucient quality to enable dose planning. Also, conversion occurred in a matter of seconds, which is line with the sCT generation time reported by IV. DISCUSSION CBCT-to-CT synthesis with a single neural network page 17 25 27,29,49 28,50 other deep learning approaches for lung , prostate and HN . We foresee these as an important step toward online ART. In conventional non-adaptive radiotherapy, this methodology can be used to evaluate the dosimetric impact of anatomical di erences occur- ring during treatment, supporting the decision to perform a rescan CT or not. In conclusion, a single cycle-GAN was successfully trained to convert CBCT to CT using unpaired training data of HN, breast and lung cancer patients. The resulted sCT resembled a diagnostic quality planning CT and featured the anatomy of the daily CBCT. In terms of dose calculation accuracy, good results were obtained for all the anatomical sites. In general, the proposed approach enables considerably fast image conversion, and it may facilitate online adaptive radiotherapy treatments. Acknowledgements We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for prototyping this research. Con ict of Interest Statement M.Maspero, M.H.F. Savenije, T.C.F. van Heijst, J.J.C. Verhoe , A.N.T. Kotte, A.C. Houwel- ing and C.A.T. van den Berg report no con ict of interests concerning the submitted work. References P. S. Cho, R. H. Johnson, and T. W. Grin, Cone-beam CT for radiotherapy applica- tions, Phys Med Biol 40, 1863{1883 (1995). J. Boda-Heggemann, F. Lohr, F. Wenz, M. Flentje, and M. Guckenberger, kV Cone- Beam CT-Based IGRT, Strahlen Onkol 187, 284{291 (2011). D. A. Ja ray, Image-guided radiotherapy: from current concept to future perspectives, Nat Rev Clin Oncol 9, 688{699 (2012). Last edited: December 25, 2019 page 18 Maspero M D. Yan, F. Vicini, J. Wong, and A. Martinez, Adaptive radiation therapy, Phys Med Biol 42, 123{132 (1997). Q. J. Wu, T. Li, Q. Wu, and F.-F. Yin, Adaptive Radiation Therapy, Cancer J 17, 182{189 (2011). J. F. Barrett and N. Keat, Artifacts in CT: Recognition and Avoidance, RadioGraph. 24, 1679{1691 (2004). R. Schulze, U. Heil, D. Gro , D. Bruellmann, E. Dranischnikow, U. Schwanecke, and E. Schoemer, Artefacts in CBCT: a review, Dentomax. Radiol. 40, 265{273 (2011). P. Remeijer, K. Deurloo, M. Eenink, K. Geleijns, J. Hermans, H. Van Herpt, M. Hol, W. De Kruijf, M. Kusters, G. D'Olieslager, M. Sijtsema, and N. Van Wieringen, NCS Report 32: Quality assurance of cone-beam CT, Technical report, Netherlands Com- mission on Radiation Dosimetry, 2019. S. Korreman, C. Rasch, H. McNair, D. Verellen, U. Oelfke, P. Maingon, B. Mijnheer, and V. Khoo, The European Society of Therapeutic Radiology and OncologyEuropean Institute of Radiotherapy (ESTROEIR) report on 3D CT-based in-room image guidance systems: A practical and technical review and guide, Radiother. Oncol. 94, 129{144 (2010). S. Ramella, M. Fiore, S. Silipigni, M. C. Zappa, M. Jaus, A. M. Alberti, P. Matteucci, E. Molfese, P. Cornacchione, C. Greco, L. Trodella, E. Ippolito, and R. M. D'Angelillo, Local Control and Toxicity of Adaptive Radiotherapy Using Weekly CT Imaging: Re- sults from the LARTIA Trial in Stage III NSCLC, J Thor Oncol 12, 1122{1130 (2017). A. Dunlop, D. McQuaid, S. Nill, J. Murray, G. Poludniowski, V. N. Hansen, S. Bhide, C. Nutting, K. Harrington, K. Newbold, and U. Oelfke, Comparison of CT number calibration techniques for CBCT-based dose calculation, Strahl. Onkol. 191, 970{978 (2015). C. Kurz, G. Dedes, A. Resch, M. Reiner, U. Ganswindt, R. Nijhuis, C. Thieke, C. Belka, K. Parodi, and G. Landry, Comparing cone-beam CT intensity correction methods for dose recalculation in adaptive intensity-modulated photon and proton therapy for head and neck cancer, Acta Oncol. 54, 1651{1657 (2015). CBCT-to-CT synthesis with a single neural network page 19 X. Zhen, X. Gu, H. Yan, L. Zhou, X. Jia, and S. B. Jiang, CT to cone-beam CT deformable registration with simultaneous intensity correction, Phys. Med. Biol. 57, 6807{6826 (2012). C. Veiga, J. McClelland, S. Moinuddin, A. Louren co, K. Ricketts, J. Annkah, M. Modat, S. Ourselin, D. D'Souza, and G. Royle, Toward adaptive radiotherapy for head and neck patients: Feasibility study on using CTtoCBCT deformable registration for dose of the day calculations, Med. Phys. 41 (2014). C. Veiga, G. Janssens, C.-L. Teng, T. Baudier, L. Hotoiu, J. R. McClelland, G. Royle, L. Lin, L. Yin, J. Metz, T. D. Solberg, Z. Tochner, C. B. Simone, J. McDonough, and B.-K. Kevin Teo, First Clinical Investigation of Cone Beam Computed Tomography and Deformable Registration for Adaptive Proton Therapy for Lung Cancer, Int. J Radiat. Oncol. Biol. Phys. 95, 549{559 (2016). G. Jarry, S. A. Graham, D. J. Moseley, D. J. Ja ray, J. H. Siewerdsen, and F. Verhae- gen, Characterization of scattered radiation in kV CBCT images using Monte Carlo simulations, Med. Phys. 33, 4320{4329 (2006). G. J. Bootsma, F. Verhaegen, and D. A. Ja ray, Ecient scatter distribution estimation and correction in CBCT using concurrent Monte Carlo tting, Med. Phys. 42, 54{68 (2014). W. Zhao, D. Vernekohl, J. Zhu, L. Wang, and L. Xing, A model-based scatter artifacts correction for cone beam CT, Med. Phys. 43, 1736{1753 (2016). M. Peroni, D. Ciardo, M. F. Spadea, M. Riboldi, S. Comi, D. Alterio, G. Baroni, and R. Orecchia, Automatic Segmentation and Online virtualCT in Head-and-Neck Adaptive Radiation Therapy, Int. J Radiat. Oncol. Biol. Phys. 84, e427{e433 (2012). C. Kurz, F. Kamp, Y.-K. Park, C. Z ollner, S. Rit, D. Hansen, M. Podesta, G. C. Sharp, M. Li, M. Reiner, J. Hofmaier, S. Neppl, C. Thieke, R. Nijhuis, U. Ganswindt, C. Belka, B. A. Winey, K. Parodi, and G. Landry, Investigating deformable image registration and scatter correction for CBCT-based dose calculation in adaptive IMPT, Med. Phys. 43, 5635{5646 (2016). Last edited: December 25, 2019 page 20 Maspero M V. Giacometti, R. B. King, C. E. Agnew, D. M. Irvine, S. Jain, A. R. Hounsell, and C. K. McGarry, An evaluation of techniques for dose calculation on cone beam computed tomography, Brit. J. Radiol. 92, 20180383 (2019). T. Niu, M. Sun, J. Star-Lack, H. Gao, Q. Fan, and L. Zhu, Shading correction for on- board cone-beam CT in radiation therapy using planning MDCT images, Med. Phys. 37, 5395{5406 (2010). Y.-K. Park, G. C. Sharp, J. Phillips, and B. A. Winey, Proton dose calculation on scatter-corrected CBCT image: Feasibility study for adaptive proton therapy, Med. Phys. 42, 4449{4459 (2015). S. Kida, T. Nakamoto, M. Nakano, K. Nawa, A. Haga, J. Kotoku, H. Yamashita, and K. Nakagawa, Cone Beam Computed Tomography Image Quality Improvement Using a Deep Convolutional Neural Network., Cureus 10, e2548 (2018). S. Xie, C. Yang, Z. Zhang, and H. Li, Scatter Artifacts Removal Using Learning-Based Method for CBCT in IGRT System, IEEE Access 6, 78031{78037 (2018). J. Maier, E. Eulig, T. V oth, M. Knaup, J. Kuntz, S. Sawall, and M. Kachelrie, Real- time scatter estimation for medical CT using the deep scatter estimation: Method and robustness analysis with respect to di erent anatomies, dose levels, tube voltages, and data truncation, Med. Phys. 46, 238{249 (2019). D. C. Hansen, G. Landry, F. Kamp, M. Li, C. Belka, K. Parodi, and C. Kurz, ScatterNet: A convolutional neural network for conebeam CT intensity correction, Med. Phys. 45, 4916{4926 (2018). X. Liang, L. Chen, D. Nguyen, Z. Zhou, X. Gu, M. Yang, J. Wang, and S. Jiang, Gen- erating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy, Phys. Med. Biol. 64, 125002 (2019). C. Kurz, M. Maspero, M. H. Savenije, G. Landry, F. Kamp, M. Pinto, M. Li, K. Parodi, C. Belka, and C. A. van den Berg, CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation, Phys. Med. Biol. 64, 225004 (2019). CBCT-to-CT synthesis with a single neural network page 21 P. Meyer, V. Noblet, C. Mazzara, and A. Lallement, Survey on deep learning for radio- therapy, Comput. Biol. Med. 98, 126{146 (2018). B. Sahiner, A. Pezeshk, L. M. Hadjiiski, X. Wang, K. Drukker, K. H. Cha, R. M. Summers, and M. L. Giger, Deep learning in medical imaging and radiation therapy, Med. Phys. 46, e1{e36 (2019). X. Han, MR-based synthetic CT generation using a deep convolutional neural network method, Med. Phys. 44, 1408{1419 (2017). M. Maspero, M. H. F. Savenije, A. M. Dinkla, P. R. Seevinck, M. P. W. Intven, I. M. Jurgenliemk-Schulz, L. G. W. Kerkmeijer, and C. A. T. van den Berg, Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy, Phys. Med. Biol. 63, 185001 (2018). J. Harms, Y. Lei, T. Wang, R. Zhang, J. Zhou, X. Tang, W. J. Curran, T. Liu, and X. Yang, Paired cycle-GAN based image correction for quantitative cone-beam CT, Med. Phys. (2019). H. C. de Boer and B. J. Heijmen, A protocol for the reduction of systematic patient setup errors with minimal portal imaging workload, Int. J Radiat. Oncol. Biol. Phys. 50, 1350{1365 (2001). G. Borgefors, Hierarchical chamfer matching: a parametric edge matching algorithm, IEEE Transactions on Pattern Analysis and Machine Intelligence 10, 849{865 (1988). D. H. Hristov and B. G. Fallone, A grey-level image alignment algorithm for registration of portal images and digitally reconstructed radiographs, Med. Phys. 23, 75{84 (1996). A. Roche, G. Malandain, X. Pennec, and N. Ayache, The correlation ratio as a new similarity measure for multimodal image registration, in Lecture Notes in Computer Science, MICCAI, pages 1115{1124, Springer, Berlin, Heidelberg, 1998. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in Proceedings of the IEEE international conference on computer vision, pages 2223{2232, 2017. Last edited: December 25, 2019 page 22 Maspero M J. M. Wolterink, A. M. Dinkla, M. H. F. Savenije, P. R. Seevinck, C. A. T. van den Berg, and I. I sgum, Deep MR to CT Synthesis Using Unpaired Data, in Simulation and Synthesis in Medical Imaging, edited by S. A. Tsaftaris, A. Gooya, A. F. Frangi, and J. L. Prince, pages 14{23, Cham, 2017, Springer International Publishing. J. Johnson, A. Alahi, and L. Fei-Fei, Perceptual Losses for Real-Time Style Transfer and Super-Resolution, Computer Vision ECCV 2016 , 694{711 (2016). P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, Image-to-Image Translation with Condi- tional Adversarial Networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5967{5976, IEEE, 2017. D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, arXiv preprint arXiv:1412.6980 (2014). D. Ulyanov, A. Vedaldi, and V. Lempitsky, Instance Normalization: The Missing Ingre- dient for Fast Stylization, arXiv preprint arXiv:1607.08022 (2016). I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, Generative Adversarial Nets, in Advances in Neural Information Processing Systems 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, pages 2672{2680, Curran Associates, Inc., 2014. D. A. Low, Gamma Dose Distribution Evaluation Tool, J. Phys. 250, 012071 (2010). M. Maspero, M. D. Tyyger, R. H. N. Tijssen, P. R. Seevinck, M. P. W. Intven, and C. A. T. van den Berg, Feasibility of magnetic resonance imaging-only rectum radio- therapy with a commercial synthetic computed tomography generation solution, Phys. Imag. Radiat. Oncol.y 7, 58{64 (2018). E. van Lin, L. van der Vight, H. Huizenga, J. Kaanders, and A. Visser, Set-up improve- ment in head and neck radiotherapy using a 3D o -line EPID-based correction protocol and a customised head and neck support, Radiother. Oncol. 68, 137{148 (2003). G. Landry, D. Hansen, F. Kamp, M. Li, B. Hoyle, J. Weller, K. Parodi, C. Belka, and C. Kurz, Comparing Unet training with three di erent datasets to correct CBCT images for prostate radiotherapy dose calculations, Phys. Med. Biol. 64, 035011 (2019). CBCT-to-CT synthesis with a single neural network page 23 Y. Li, J. Zhu, Z. Liu, J. Teng, Q. Xie, L. Zhang, X. Liu, J. Shi, and L. Chen, A preliminary study of using a deep convolution neural network to generate synthesized CT images based on CBCT for adaptive radiotherapy of nasopharyngeal carcinoma, Phys. Med. Biol. 64 (2019). C. Paganelli, G. Meschini, S. Molinelli, M. Riboldi, and G. Baroni, Patient-speci c validation of deformable image registration in radiation therapy: Overview and caveats, Med. Phys. 45, e908{e922 (2018). Last edited: December 25, 2019 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Electrical Engineering and Systems Science arXiv (Cornell University)

CBCT-to-CT synthesis with a single neural network for head-and-neck, lung and breast cancer adaptive radiotherapy

Loading next page...
 
/lp/arxiv-cornell-university/cbct-to-ct-synthesis-with-a-single-neural-network-for-head-and-neck-g1p2j51U4Z
ISSN
2405-6316
eISSN
ARCH-3348
DOI
10.1016/j.phro.2020.04.002
Publisher site
See Article on Publisher Site

Abstract

Purpose: CBCT-based adaptive radiotherapy requires daily images for accurate dose calculations. This study investigates the feasibility of applying a single convolutional network to facilitate CBCT-to-CT synthesis for head-and-neck, lung and breast cancer patients. Methods: Ninety-nine patients diagnosed with head-and-neck, lung or breast cancer undergoing radiotherapy with CBCT-based position veri cation were included in this study. CBCTs were registered to planning CTs according to clinical procedures. Three cycle-consistent generative adversarial networks (cycle-GANs) were trained in an un- paired manner on 15 patients per anatomical site generating synthetic-CTs (sCTs). Another network was trained with all the anatomical sites together. Performances of all four networks were compared and evaluated for image similarity against rescan CT (rCT). Clinical plans were recalculated on CT and sCT and analysed through voxel- based dose di erences and -analysis. Results: A sCT was generated in 10 seconds. Image similarity was comparable be- tween models trained on di erent anatomical sites and a single model for all sites. Mean dose di erences < 0:5% were obtained in high-dose regions. Mean gamma (2%,2mm) pass-rates > 95% were achieved for all sites. Conclusions: Cycle-GAN reduced CBCT artefacts and increased HU similarity to CT, enabling sCT-based dose calculations. The speed of the network can facilitate on- line adaptive radiotherapy using a single network for head-and-neck, lung and breast cancer patients. arXiv:1912.11136v1 [physics.med-ph] 23 Dec 2019 Contents I. Introduction 1 II. Material and methods 2 II.A. Imaging protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 II.B. CBCT-to-CT synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 II.B.1. Image pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 II.B.2. Network architecture and training . . . . . . . . . . . . . . . . . . . . 4 II.B.3. Image post-processing . . . . . . . . . . . . . . . . . . . . . . . . . . 6 II.C. Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 II.C.1. Image comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 II.C.2. Dose comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 III. Results 8 III.A. Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 III.B. Image comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 III.C. Dose comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 IV. Discussion 15 Acknowledgements 17 Con ict of Interest Statement 17 References 17 ii CBCT-to-CT synthesis with a single neural network page 1 I. Introduction In modern external beam image-guided radiotherapy (IGRT), cone-beam computed tomo- 1,2,3 graphy (CBCT) plays a crucial role in accurate patient position veri cation . Also, CBCT 4,5 can facilitate adaptive radiotherapy (ART) by visualising daily anatomical variations . CBCT image quality is inferior to that of CT in soft-tissue contrast and Houns eld Units 6,7,8 (HU) consistency due to the presence of artefacts . Therefore, CBCT is not sucient to perform accurate dose calculations and patients need to be referred for a rescan CT (rCT) whenever anatomical di erences are noted between daily images and planning CT . However, scheduling and acquiring a rCT adds logistic complexity and patient burden to the treatment. On the contrary, with ART these issues can be addressed by exploiting the daily CBCT images to reduce set-up errors and eliminate the need for an rCT . A prerequisite for online ART is that the CBCT quality and HU accuracy is sucient to enable dose calculation. Considerable literature has recently emerged proposing to correct CBCT imaging arte- 11,12 facts and increase image intensity consistency using: look-up table-based approaches , deformable imaging registration (DIR) of the planning CT to the daily anatomy on 13,14,15 CBCT and model- or Monte Carlo-based methods for scatter estimation and correc- 16,17,18 19 tion . Speci cally, DIR enabled accurate dose calculations for head-and-neck (HN) but obtained lower dose accuracy in more complex anatomical changes such as lung and 20,21 20,22,23 pelvis . Also, Monte Carlo-based methods were suitable for ART . These tech- niques can be deployed on a time scale of minutes, which is not acceptable when aiming to use CBCT images for daily online dose evaluation or online pre-treatment adaptation. Recently, deep learning has been proposed for fast CBCT artefact correc- 24,25,26,27,28,29 tion . Deep learning is a branch of arti cial intelligence and machine learning that involves the use of neural networks to generate a hierarchical representation of the in- 30,31 put data to achieve a speci c task without the need of hand-engineered features . Deep learning has shown promising results solving image-to-image translation problems within sec- 32,33 onds . In this sense, previous work demonstrated the use of a two-dimensional (2D) U-net 24,25,26 to improve CBCT image quality . Moreover, it has been shown that converting CBCT 27,29 with deep learning resulted in accurate dose calculation for prostate cancer patients and Last edited: December 25, 2019 page 2 Maspero M 28,34 HN cancer patients . In this study, we investigate whether CBCTs converted with convolutional networks may be used as a surrogate of the daily anatomy for dose calculations. We employ a network trained in an unpaired manner to convert CBCT-to-CT of HN, lung, or breast cancer patients investigating whether a single network can generalise for the three anatomical sites. A single network trained for all the anatomical sites was compared to three networks trained per anatomical site. Performances of all four networks were compared and evaluated for image similarity and dose calculation accuracy between CT and rCT. II. Material and methods II.A. Imaging protocols Ninety-nine patients diagnosed with HN (33), lung (33) or breast (33) cancer undergoing radiotherapy were retrospectively included in this study. Irradiations were performed be- tween May 2016 and February 2019 on Agility linacs (Elekta AB, Sweden) with CBCT-based pre-treatment position veri cation. An rCT was acquired in case anatomical variations were noted on the CBCT. We included at least fourteen patients with rCT per site. The (r)CTs were acquired on a Brilliance Big Bore (Philips Healthcare, Ohio, USA); CBCTs were acquired using X-ray volumetric imaging (XVI, v5.0.2b72 Elekta AB, Sweden) system. Table 1 reports the imaging protocols for CT, rescan CT (rCT) and CBCT for all the patients included in the study. CBCTs were acquired with 0.25 rotation/s gantry speed and 5.5 frames/s. All the CBCTs were acquired with a 200 -arc utilising an empty lter cassette (F0) in combination with a centred detector panel (S position, maximum FOV=27x27 cm ). The eld-of-view (FOV) was in four cases (elective lymph-nodes irradiations or double-sided irradiation for breast and HN patients) enlarged to a maximum of 41x41 cm using a shifted detector panel (M position) to accommodate the CTV in the CBCT FOV. Imaging frequency of CBCT followed the extended non-action limit protocol : online corrections (action level 0 mm) were applied in the case of partial or ablative breast irradia- tion, and oine long (N=3, P=5) and short (N=2, P=3) scheme were applied for irradiations II. MATERIAL AND METHODS CBCT-to-CT synthesis with a single neural network page 3 Table 1: Overview of CT (including also rescan (r)CT) and CBCT imaging protocols in terms of eld-of-view (FOV), acquisition matrix (Acq matrix), resolution (Res), tube voltage (kVp), exposure (ms) and current (mA). For exposure and current, the mean value () was reported along with the range. a a b b b FOV Res Voltage Exposure Current Modality Site Acq matrix 3 3 [cm ] [mm ] [kVp] [ms] [mA] 43-70 512 0.83-1.37 98365 15950 Head-and-neck 43-70 512 0.83-1.37 120 923-1090 47-271 30-111 101-535 2-3 47-70 512 0.92-1.37 1050109 6337 (r)CT Breast 47-70 512 0.92-1.37 120 923-1332 31-271 31-120 103-400 2-3 29-70 512 0.57-1.37 38863095 9866 Lung 29-70 512 0.57-1.37 120 500-10091 30-271 23-220 76-660 1-3 27 135-270 1-2 115 143 Head-and-neck 27 135-270 1-2 100 10-40 10-20 13-53 126-526 1-2 27-41 270-540 0.5-1 332 172 CBCT Breast 27-41 270-540 0.5-1 120 32-40 16-20 26-53 262-526 0.5-1 27 270 1-2 316 201 Lung 27 270 1-2 120 10-40 16-25 26-53 128-528 1-2 Expressed in RL, AP, FH directions; the range is reported in terms of min-max. Reported in terms of mean value and range=min-max. Except for H18 and H20 where kVP was 120. Except for L11, L22 and L24 where kVP was 100. having > 20 and < 20 fractions, respectively. Imaging frequency may have been increased after consultation between a medical physicist and a radiotherapist on a single patient-basis in case large inter-fraction motions were observed in the initial fractions or whenever RT technicians reported diculties in reproducing the planning position. CBCTs were translated to apply clinical set-up corrections and resampled to the plan- ning CT within the X-ray volumetric imaging (XVI, v5.0.2b72 Elekta AB, Sweden) system. Registrations were estimated within a clip-box including the CTV based on bone rigid (trans- lation and rotation) matching . For the breast patients treated with local RT followed by a sequential boost, a dual rigid registration was performed based rst on bone matching 37,38 followed by grey level (soft-tissue) matching . The centre of rotation was assigned as the centre of the PTV. Last edited: December 25, 2019 II.A. Imaging protocols page 4 Maspero M II.B. CBCT-to-CT synthesis II.B.1. Image pre-processing Before supplying images to the network, CT and CBCT were cropped to the size of the CBCT FOV after identifying the so-called \Mask " according to the following steps. CBCTs were CBCT thresholded at -999.9, obtaining a binary mask. In each transverse slice containing the binary mask, morphological closure was performed, and the smallest bounding box containing the mask was found. The biggest circle contained in the bounding box was searched starting from a radius of 26.9 cm and iteratively increasing its size. The circle was propagated for all the slices obtaining Mask . CT and CBCT were cropped in the bounding box containing CBCT Mask . CBCT In addition to cropping, voxel intensity of CT and CBCT were clipped within the interval [1000;3071] HU and image intensity was linearly rescaled to 16-bit. II.B.2. Network architecture and training To generate CT from CBCT, a 2D cycle-generative adversarial network (cycle-GAN) was adopted . Cycle-GANs enable unpaired training, which, compared to paired training, makes the network less sensitive to residual mismatch of CT and CBCT . The network consisted of two cycles called \forward" and \backwards" during which GANs generated CT from CBCT and vice-versa. Moreover, so-called \cycle-consistency" was enforced with an L -norm such that after converting from CBCT to CT and vice-versa, the original image should be obtained. The architecture, based on the cycle-GAN provided by 39 41 Zhou et al. , was implemented in Tensor ow (v1.3.0). Nine-blocks residual networks were employed as generators and Patch-GANs as discriminators (Figure 1). Stochastic gradient descendent was used applying an Adam solver with learning rate = 0.0002, momentum parameters = 0.5 and = 0.999. Instance normalisation was employed with a batch 1 2 size of 1. The weights of the network were randomly initialised from N (0; 0:02). Weight op- timisation was performed as in Goodfellow et al. alternating between one gradient descen- dent step on the discriminator network and one step on the generator network after having performed a forward and backward cycle. A structured loss function GAN+  L +cycle- consistency with  = 25 was adopted. The original implementation of the network by Zhou II. MATERIAL AND METHODS II.B. CBCT-to-CT synthesis CBCT-to-CT synthesis with a single neural network page 5 Figure 1: Architecture of the nine-block residual network used as a generator (top) and of the convolutional network called Patch-GAN used as a discriminator. The size of the images is numerically reported, except for the residual block, where it remains stable. Note that the nine-blocks are omitted in the schematic. Each of the lters had stride two, kernel size four; leaky rectilinear recti er unit had a scalar multiplier of 0.2, padding was applied in re ect mode. et al was modi ed to accommodate 16-bit grey-scale images with a size of 256x256. To train, validate and test the network, patients were split into three datasets: 15 patient per site for training, 8 for validation and the remaining 10 for test. The validation set was used to aid hyperparameter optimisation and to determine at which iteration the https://github.com/xhujoy/CycleGAN-tensorflow Last edited: December 25, 2019 II.B. CBCT-to-CT synthesis page 6 Maspero M training could be stopped to avoid over- tting (early-stopping), while the test set was used to evaluate the performance of the network. To investigate the impact on dose calculation of a di erent CT, the patients included in the test set were selected among the patients with an rCT and with CBCT and rCT acquired with minimal time di erences. Patients' demographics were controlled to ensure data balancing in terms of the number of patients in the three sets. Also, we inspected the ratio of male/female, distribution age, tumour staging distribution and linac on which CBCT were acquired (Supplementary Material). Training of the cycle-GAN was performed in the transverse plane, and for each iteration, random CT, and CBCT slices of di erent patients were supplied. Three networks were trained separately on each anatomical site. Another network was trained on all anatomical sites to investigate whether a single network may generalise for all anatomical sites. The networks were trained for 200 epochs on a Tesla P100 (16 Gb, NVIDIA, California, USA) graphical processing unit (GPU) with batch size one and image pool of 1000 images. Data augmentation was applied during training by ipping the images left and right and randomly cropping of 30x30 voxels after having bi-linearly resampled the images to 286x286 voxels in Mask . Early stopping was applied selecting the earliest epoch for which average L in CBCT 1 the body contour over the pts of the validation test was the lowest and within one  from L calculated at every 10 epoch (a total of 20 models were stored, one each 10 epochs). The total amount of slices utilised during training was reported to verify data balancing among sites: 3668 transverse slices acquired in a minimum of four di erent linacs were used for training, composed by 1606, 1046 and 1016 slices from HN, lung and breast cancer patients, respectively. II.B.3. Image post-processing First, the trained model was applied within Mask to the pre-processed CBCTs (as de- CBCT scribed in II.B.1.) obtaining 16-bit images. Then, the HU intensity range of [-1000;3071] was restored with a linear rescaling obtaining the so-called CBCT . CBCT were conv conv bi-linearly resampled from a matrix size of 256x256 to the original CBCT resolution. To generate images for the full CT FOV, the CBCT was substituted in the original conv CT within the Mask . The image obtained combining CT and CBCT will be referred CBCT conv to as synthetic-CT (sCT) (Figure 2). II. MATERIAL AND METHODS II.B. CBCT-to-CT synthesis CBCT-to-CT synthesis with a single neural network page 7 Figure 2: Schematic of the image work ow for a 2D transverse slice of a breast cancer patient. After image acquisition, registration (1) and pre-processing (2) the trained network is deployed producing converted CBCT (CBCT , 3) which substituted the original CT within Mask conv CBCT obtaining the so-called synthetic CT (sCT). II.C. Evaluation Image evaluation in terms of similarity between sCT and rCT was performed to assess whether the single network trained with all the anatomical sites was comparable to the three networks trained per anatomical site. If performances were comparable, the single network was considered to assess the appropriateness of the CBCT conversion with the trained cycle-GAN on the test set with an image and a dose comparison. II.C.1. Image comparison Similarities between the image intensity of sCT, CBCT, CT and rCT were calculated within Mask in terms of mean absolute error (MAE) and mean error (ME) as proposed by Liang CBCT et al. . Rescan CT was considered as ground truth, and the metrics were calculated in terms of mean 1 and range. Wilcoxon signed-rank tests were conducted between sCT/rCT and CT/rCT for MAE. Additional metrics are reported in Supplementary Material. Last edited: December 25, 2019 II.C. Evaluation page 8 Maspero M II.C.2. Dose comparison For the patients in the test sets, clinical plans were recalculated on CT, sCT and rCT images in Monaco (v 5.11.02, Elekta AB, Sweden) using a Monte Carlo algorithm on a grid of 3 mm with 5% and 3% statistical uncertainty for volumetric modulated arc therapy (VMAT) and intensity-modulated radiotherapy (IMRT) plans, respectively. Clinical contours, delineated by a radiation oncologist on the planning CT, were rigidly transferred to the sCT and rCT except for the body contour, which was automatically re-delineated. These contours were considered as volumes of interest (VOIs). sCTrCT Dose distributions were analysed through relative dose di erences (DD = sCT rCT CTrCT and DD = ) in the high dose region (dose > 90% of the prescribed dose). Also, CT rCT 3D -analysis with 3%,3mm and 2%,2mm criteria relative to dose on rCT within regions of 50% prescription dose were performed. For all dose comparisons, a 15 mm cropping in the proximity of body contour was performed to take account of dose build-up in the proximity of the skin . To investigate the impact of dose di erence within VOIs, analysis of dose-volume his- togram (DVH) points was performed on sCT and rCT. The DVH points analysed were the maximum dose and mean dose. OARs were considered for such analysis when they were present in at least four of the patients for each anatomical site: submandibular and parotid glands, spinal cord, larynx and brain stem for head-and-neck patients; lungs, heart, oesoph- agus, humerus and spinal cord for breast patients; lungs, heart, oesophagus, spinal cord and trachea for lung patients. III. Results III.A. Network Cycle-GANs required about eight days and ve hours training on a GPU Tesla P100 (NVIDIA Corporation) employing 200 epochs with 3668 slices. As a result of the early stopping investigation, after inspecting the L loss function on the training and validation set, we opted for utilising 160000 (100 epochs), 180000 (160 epochs), 180000 (170 epochs) iterations for the network trained on HN, breast and lung dataset, respectively, and III. RESULTS CBCT-to-CT synthesis with a single neural network page 9 360000 iterations (100 epochs) for the network trained on all the three sites combined. Generating sCT required <10 s for an entire CBCT volume (70 slices) on GPU and about 40 s on CPU. III.B. Image comparison The time between rCT and CBCT in the test set was on average ( [min; max]) 1  3 [0;8] days and 29 11 [8;67] days between CT and CBCT. The increased time between CT and CBCT may result in larger di erences when comparing rCT vs CT compared to sCT vs rCT. Similarity metrics over the test patients are reported in Table 2. Table 2: Overview of the image comparison. Image comparison calculated as mean (1) and range ([min;max]) of the test dataset (30 patients) compared to the reference dataset in terms of mean absolute error (MAE) and mean error (ME) between the Test image minus the Ref image. Site Head-and-Neck Breast Lung MAE ME MAE ME MAE ME Test Ref [HU] [HU] [HU] [HU] [HU] [HU] 19520 -12233 15240 7137 21944 15348 CBCT rCT [160;230] [-183;-71] [98;213] [7;115] [133;280] [94;230] sCT single 5312 -37 6618 -613 8310 -211 rCT network [37;77] [-15;10] [41;95] [-24;13] [72;104] [-25;10] sCT separate 5112 -66 6718 -511 869 -514 rCT networks [35;74] [-16;4] [41;98] [-18;14] [73;105] [-28;10] 6317 -1815 6324 820 9423 922 CT rCT [-40;90] [-46;3] [40;115] [-14;54] [68;146] [-33;36] sCT obtained from a single network trained on all the anatomical sites. sCT obtained from three di erent networks trained on each anatomical site. Generic network vs site-speci c networks No statistically signi cant di erences (p>0.35) were found between networks trained per separate anatomical site and the single network trained with all three anatomical sites. This justi es the use of the single model trained for all the anatomical sites for assessing the accuracy of HU and for the dose comparison. Accuracy of HU One can notice that similarity increased between sCT and rCT compared to CBCT and rCT; e.g. MAE decreased from 19520 (CBCT/rCT) to 5312 HU (sCT/rCT) for HN. All the similarity metrics calculated between sCT/rCT and CT/rCT can be considered equivalent Last edited: December 25, 2019 III.B. Image comparison page 10 Maspero M to the metrics calculated between CT and rCT, with no signi cant di erences (p>0.14) for all the three anatomical sites. The mean MAE and range for sCT/rCT were smaller than for CT/rCT due to the reduced time between sCT/rCT, which resulted in less anatomical di erences. Figure 3, 4, 5 show examples of CBCT and sCT obtained from the single network for a HN, breast and lung cancer patient, respectively. One can observe that the network reduced scatter artefacts while retaining anatomical accuracy. Considering an example of lung patient (Figure 5), one can observe the occurrence of atelectasis between CT and rCT/sCT. III.C. Dose comparison Figure 3, 4, 5 report also dose distributions calculated on CT, rCT and sCT along with their DVHs. One can qualitatively notice small di erence between doses on rCT and sCT. When considering the quantitative results, no signi cant di erences were observed between DD and DD . On average (Table 3), dose di erences between sCT/rCT (DD ) sCT CT sCT were lower then for CT/rCT (DD ), e.g. in the high dose region (D> 90%) maximum rCT mean di erences in the range [0.1;0.2]% and [-0.3;0.9]% were found for DD and DD , sCT rCT respectively. Table 3: Statistics of the dose comparison of the thirty patients in the test set. The values are reported as percentage mean1 and range [min; max]. sCT vs rCT CT vs rCT 1 2 3 4 3 4 DD DD 3%;3 mm 2%;2 mm 3%;3 mm 2%;2 mm sCT rCT Sites [%] [%] [%] [%] [%] [%] 0:1 0:5 98:4 1:9 95:3 2:8 0:9 1:1 97:0 2:3 91:1 4:9 Head-and-neck [0:8; 0:7] [93:4; 100] [90:1; 99:6] [1:8; 2:0] [92:7; 99:6] [80:7; 97:4] 0:1 0:4 96:5 4:1 89:4 8:2 0:3 0:8 95:7 5:5 88:8 9:2 Breast [0:5; 0:8] [86:1; 99:6] [70:8; 97:8] [1:7; 0:7] [81:1; 99:5] [66:3; 97:5] 0:2 0:9 97:0 2:9 91:3 6:2 0:1 1:5 95:2 3:6 87:7 6:8 Lung [1:3; 1:8] [91:5; 99:7] [82:3; 98:4] [2:6; 3:0] [89:8; 99:4] [79:4; 96:6] c sCTrCT DD =  100 on dose > 90% of the prescribed dose. sCT rCT rCTCT DD =  100 on dose >90% of the prescribed dose. rCT CT Pass rates of on dose > 50% of the prescribed dose. 3%;3 mm Pass rates of on dose > 50% of the prescribed dose. 2%;2 mm The mean gamma pass rates with the 2%,2mm criteria were higher for sCT/rCT com- pared to CT/rCT for all VOIs, which is in line with the dose di erences observed. All DVH III. RESULTS III.C. Dose comparison CBCT-to-CT synthesis with a single neural network page 11 points di ered on average <0.5% compared to rCT. DVH points di erences were < 2% ex- cept for the heart of a breast patient (B31, -5.6%), the oesophagus of two breast patients (B30, 3.1% and B31, 2.3%), left lung and spinal cord of two lung patients (L25, -2.1% and L27, 3.8%, respectively). Images of the patients with doses di erences in VOIs > 2% were inspected on a single-case basis, as reported in the Supplementary Material. We noticed that large dose di erences were in low-dose regions, which are more sensitive to statistical di erences due to the low amount of events in the Monte Carlo dose calculations. For a lung case (L25), anatomical di erences were reported as the cause of the observed di erences. Also, residual artefacts characterised by inhomogeneous HUs seem to be present along the craniocaudal direction in the lungs for sCT; it appears that for this case the CBCT artefacts were not fully recovered by the network within the lungs. Also for the other lung case (L27), anatomical di erences were observed in the lung. In addition, we noticed the patient was obese and the CBCTs were characterised by severe scatter artefacts. On sCT, the spinal cord was not entirely recovered, possibly resulting in local di erence. Besides, the spinal cord is located in a low-dose region, which may be highlighted when considering metrics as voxel-wise relative di erences. Last edited: December 25, 2019 III.C. Dose comparison page 12 Maspero M H24 sCT , rCT ; Presc dose = 70 Gy 20161019 20161020 CTVp5425 CTVnR5425 SpinalCord ParotidR ParotidL SubmandR SubmandL Larynx Brainstem body 0 20 40 60 80 100 Dose [% to Presc dose] st Figure 3: Sagittal views for the head-and-neck cancer patient H24 of: (1 row) CBCT st nd rd th (1 column), CT (2 column), rescan CT (rCT, 3 column) and synthetic CT (sCT, 4 nd rd column), along with (2 row) the respective di erence to rCT, the doses (3 row). The red, black, or green dotted rectangles indicate the position of Mask . The days refer to CBCT th the acquisition date of the rCT. In the 4 row, the DVH is shown for target and OARs of sCT (continuous lines) and rCT (dashed lines). III. RESULTS III.C. Dose comparison Volume [%] CBCT-to-CT synthesis with a single neural network page 13 B27 sCT , rCT ; Presc dose = 61.18 Gy 20171024 20171025 CTV4669 CTVb6118 CTVnL14669 80 LongL LongR TotalLung Esophagus Heart SpinalCord PRVhumerus 40 body 0 50 100 150 Dose [% to Presc] st st Figure 4: Coronal views for the breast cancer patient B27 of: (1 row) CBCT (1 column), nd rd th CT (2 column), rescan CT (rCT, 3 column) and synthetic CT (sCT, 4 column), along nd rd with (2 row) the respective di erence to rCT, the doses (3 row). The red, black, or green dotted rectangles indicate the position of Mask . The days refer to the acquisition date of CBCT th the rCT. In the 4 rows, the DVH is shown for target and OARs of sCT (continuous lines) and rCT (dashed lines). Last edited: December 25, 2019 III.C. Dose comparison Volume [%] page 14 Maspero M L26 sCT , rCT ; Presc dose = 30 Gy 20170926 20170926 ITV3000 Longptv LongL LongR Esophagus Heart SpinalCord05 Trachea body 0 50 100 150 Dose [% to Presc] st st Figure 5: Axial views for the lung cancer patient L26 of: (1 row) CBCT (1 column), CT nd rd th (2 column), rescan CT (rCT, 3 column) and synthetic CT (sCT, 4 column), along with nd rd (2 row) the respective di erence to rCT, the doses (3 row). The red, black, or green dotted rectangles indicate the position of Mask . The days refer to the acquisition date of CBCT th the rCT. In the 4 row, the DVH is shown for target and OARs of sCT (continuous lines) and rCT (dashed lines). III. RESULTS III.C. Dose comparison Volume [%] CBCT-to-CT synthesis with a single neural network page 15 IV. Discussion Cycle-consistent generative adversarial network (cycle-GAN) increased the accuracy of HU in CBCT, enabling sCT-based calculations for HN, lung and breast cancer patients. Also, we found that a single network trained on all the three sites performed similarly to three networks trained on each anatomical site. When investigating the accuracy of HU on sCT calculating image similarity to rescan CT, we found that HU values were comparable to values observed between CT and rCT. We observed a slight increase in performance for HN compared to breast and lung cancer patients. The network was trained with higher amount of slices for HN (1606) compared to lung and breast (1046 and 1016, respectively). We hypothesise that this data imbalancing may have resulted in relatively increased perfomances for HN cancer patients. Also, the use of immobilisation masks for HN case may increase the reproducibility of patient set- up or reduce motion artefacts in the images (both CT and CBCT) . Though variations in the CBCT imaging protocol were reported, e.g. kV, mAs and linac where the images were acquired, we did not observe any e ect on the quality of sCT. It may be of interest to investigate thoroughly the in uence to the robustness of the method to variations of acquisition settings, as already proposed by Maier et al. . In terms of dose calculation accuracy, we compared sCT to rCT, achieving excellent results for all the anatomical sites. Previous work with deep learning was performed on 24,27,29,49 28,50 25 prostate , HN and lung patients. For HN patients, similar ndings were re- ported by Liang et al. , where also a cycle-GAN was utilised where a mean (1) 2%, 2mm pass-rates of 98.41.7% was obtained compared to 98.41:9% of this work. Also, Li et al. used a 2D U-net with residual convolutional units achieving mean DVH point di erence < 1% . In our study, similar mean DVH point di erences (< 0:5%) were achieved, which demonstrates the high sCT quality achieved with our approach. For lung patients, Xie et al. applied patch-based residual learning on lung patients obtaining a conspicuous correction of cupping and streaking artefacts . Unfortunately, they did not perform any dose calculations and used di erent metrics, making it dicult to compare the studies. Repositioning inevitably occurred between CBCT, rCT and CT. To further minimise anatomical and set-up di erences, we could have recurred to deformable image registration Last edited: December 25, 2019 page 16 Maspero M (DIR) to increase the similarity of CBCT/sCT and CT/rCT. However, we opted against it for the following reasons: (i) since we were trying to reproduce the dose derived by CT- based calculations, we did not want to modify CT or rCT further; (ii) residual deformation errors should be thoroughly evaluated , and this was deemed out of the scope of this investigation; (iii) recurring to using solely translation mimics the set-up procedure that is currently performed clinically at the linacs, and we aimed at observing the impact of dose evaluation in a comparable setting. The main limitation of this study is deemed to be the cohort size: ten patients per anato- mical sites in the test set may be considered as a low number. Before clinical implementation, a study including a larger number of patients should be initiated, paying particular attention to the data variability and data balancing among anatomical sites. Besides, we did not adapt the contours of targets and OARs, which is necessary to investigate the clinical impact of replanning thoroughly. To our knowledge, this is the most extensive study so far presented utilising a convolutional neural network for sCT generation with ninety-nine included pa- tients. Also, notwithstanding the relatively limited sample, this work o ers valuable insights into the generalisation capability of a single cycle-GAN, and, in general, in showing that a single neural network can convert CBCTs of multiple sites. Currently, we balanced the sites based on the number of patients performing training with about 1.5 times more images for HN compare to lung and breast patients. It would be interesting to investigate in a future study whether di erent balancing may maintain com- parable image similarity for all the anatomical sites. We believe that further improvements can be made by balancing data in terms of the number of slices included in the training. In our study, we showed, for the rst time, that a single cycle-GAN can be utilised for multiple anatomical sites as HN, breast and lung. This nding has important implications for simplifying the training of a convolutional network since a single network may be adopted for di erent anatomical sites. To fully understand whether a single network may facilitate CBCT-based dose calculations for the whole body, we are currently performing a novel study including additional anatomical areas, e.g. pelvis, lower abdomen and brain. The impact of our work is that with a single cycle-GAN CBCTs were converted into CTs, resulting in sCTs that have sucient quality to enable dose planning. Also, conversion occurred in a matter of seconds, which is line with the sCT generation time reported by IV. DISCUSSION CBCT-to-CT synthesis with a single neural network page 17 25 27,29,49 28,50 other deep learning approaches for lung , prostate and HN . We foresee these as an important step toward online ART. In conventional non-adaptive radiotherapy, this methodology can be used to evaluate the dosimetric impact of anatomical di erences occur- ring during treatment, supporting the decision to perform a rescan CT or not. In conclusion, a single cycle-GAN was successfully trained to convert CBCT to CT using unpaired training data of HN, breast and lung cancer patients. The resulted sCT resembled a diagnostic quality planning CT and featured the anatomy of the daily CBCT. In terms of dose calculation accuracy, good results were obtained for all the anatomical sites. In general, the proposed approach enables considerably fast image conversion, and it may facilitate online adaptive radiotherapy treatments. Acknowledgements We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for prototyping this research. Con ict of Interest Statement M.Maspero, M.H.F. Savenije, T.C.F. van Heijst, J.J.C. Verhoe , A.N.T. Kotte, A.C. Houwel- ing and C.A.T. van den Berg report no con ict of interests concerning the submitted work. References P. S. Cho, R. H. Johnson, and T. W. Grin, Cone-beam CT for radiotherapy applica- tions, Phys Med Biol 40, 1863{1883 (1995). J. Boda-Heggemann, F. Lohr, F. Wenz, M. Flentje, and M. Guckenberger, kV Cone- Beam CT-Based IGRT, Strahlen Onkol 187, 284{291 (2011). D. A. Ja ray, Image-guided radiotherapy: from current concept to future perspectives, Nat Rev Clin Oncol 9, 688{699 (2012). Last edited: December 25, 2019 page 18 Maspero M D. Yan, F. Vicini, J. Wong, and A. Martinez, Adaptive radiation therapy, Phys Med Biol 42, 123{132 (1997). Q. J. Wu, T. Li, Q. Wu, and F.-F. Yin, Adaptive Radiation Therapy, Cancer J 17, 182{189 (2011). J. F. Barrett and N. Keat, Artifacts in CT: Recognition and Avoidance, RadioGraph. 24, 1679{1691 (2004). R. Schulze, U. Heil, D. Gro , D. Bruellmann, E. Dranischnikow, U. Schwanecke, and E. Schoemer, Artefacts in CBCT: a review, Dentomax. Radiol. 40, 265{273 (2011). P. Remeijer, K. Deurloo, M. Eenink, K. Geleijns, J. Hermans, H. Van Herpt, M. Hol, W. De Kruijf, M. Kusters, G. D'Olieslager, M. Sijtsema, and N. Van Wieringen, NCS Report 32: Quality assurance of cone-beam CT, Technical report, Netherlands Com- mission on Radiation Dosimetry, 2019. S. Korreman, C. Rasch, H. McNair, D. Verellen, U. Oelfke, P. Maingon, B. Mijnheer, and V. Khoo, The European Society of Therapeutic Radiology and OncologyEuropean Institute of Radiotherapy (ESTROEIR) report on 3D CT-based in-room image guidance systems: A practical and technical review and guide, Radiother. Oncol. 94, 129{144 (2010). S. Ramella, M. Fiore, S. Silipigni, M. C. Zappa, M. Jaus, A. M. Alberti, P. Matteucci, E. Molfese, P. Cornacchione, C. Greco, L. Trodella, E. Ippolito, and R. M. D'Angelillo, Local Control and Toxicity of Adaptive Radiotherapy Using Weekly CT Imaging: Re- sults from the LARTIA Trial in Stage III NSCLC, J Thor Oncol 12, 1122{1130 (2017). A. Dunlop, D. McQuaid, S. Nill, J. Murray, G. Poludniowski, V. N. Hansen, S. Bhide, C. Nutting, K. Harrington, K. Newbold, and U. Oelfke, Comparison of CT number calibration techniques for CBCT-based dose calculation, Strahl. Onkol. 191, 970{978 (2015). C. Kurz, G. Dedes, A. Resch, M. Reiner, U. Ganswindt, R. Nijhuis, C. Thieke, C. Belka, K. Parodi, and G. Landry, Comparing cone-beam CT intensity correction methods for dose recalculation in adaptive intensity-modulated photon and proton therapy for head and neck cancer, Acta Oncol. 54, 1651{1657 (2015). CBCT-to-CT synthesis with a single neural network page 19 X. Zhen, X. Gu, H. Yan, L. Zhou, X. Jia, and S. B. Jiang, CT to cone-beam CT deformable registration with simultaneous intensity correction, Phys. Med. Biol. 57, 6807{6826 (2012). C. Veiga, J. McClelland, S. Moinuddin, A. Louren co, K. Ricketts, J. Annkah, M. Modat, S. Ourselin, D. D'Souza, and G. Royle, Toward adaptive radiotherapy for head and neck patients: Feasibility study on using CTtoCBCT deformable registration for dose of the day calculations, Med. Phys. 41 (2014). C. Veiga, G. Janssens, C.-L. Teng, T. Baudier, L. Hotoiu, J. R. McClelland, G. Royle, L. Lin, L. Yin, J. Metz, T. D. Solberg, Z. Tochner, C. B. Simone, J. McDonough, and B.-K. Kevin Teo, First Clinical Investigation of Cone Beam Computed Tomography and Deformable Registration for Adaptive Proton Therapy for Lung Cancer, Int. J Radiat. Oncol. Biol. Phys. 95, 549{559 (2016). G. Jarry, S. A. Graham, D. J. Moseley, D. J. Ja ray, J. H. Siewerdsen, and F. Verhae- gen, Characterization of scattered radiation in kV CBCT images using Monte Carlo simulations, Med. Phys. 33, 4320{4329 (2006). G. J. Bootsma, F. Verhaegen, and D. A. Ja ray, Ecient scatter distribution estimation and correction in CBCT using concurrent Monte Carlo tting, Med. Phys. 42, 54{68 (2014). W. Zhao, D. Vernekohl, J. Zhu, L. Wang, and L. Xing, A model-based scatter artifacts correction for cone beam CT, Med. Phys. 43, 1736{1753 (2016). M. Peroni, D. Ciardo, M. F. Spadea, M. Riboldi, S. Comi, D. Alterio, G. Baroni, and R. Orecchia, Automatic Segmentation and Online virtualCT in Head-and-Neck Adaptive Radiation Therapy, Int. J Radiat. Oncol. Biol. Phys. 84, e427{e433 (2012). C. Kurz, F. Kamp, Y.-K. Park, C. Z ollner, S. Rit, D. Hansen, M. Podesta, G. C. Sharp, M. Li, M. Reiner, J. Hofmaier, S. Neppl, C. Thieke, R. Nijhuis, U. Ganswindt, C. Belka, B. A. Winey, K. Parodi, and G. Landry, Investigating deformable image registration and scatter correction for CBCT-based dose calculation in adaptive IMPT, Med. Phys. 43, 5635{5646 (2016). Last edited: December 25, 2019 page 20 Maspero M V. Giacometti, R. B. King, C. E. Agnew, D. M. Irvine, S. Jain, A. R. Hounsell, and C. K. McGarry, An evaluation of techniques for dose calculation on cone beam computed tomography, Brit. J. Radiol. 92, 20180383 (2019). T. Niu, M. Sun, J. Star-Lack, H. Gao, Q. Fan, and L. Zhu, Shading correction for on- board cone-beam CT in radiation therapy using planning MDCT images, Med. Phys. 37, 5395{5406 (2010). Y.-K. Park, G. C. Sharp, J. Phillips, and B. A. Winey, Proton dose calculation on scatter-corrected CBCT image: Feasibility study for adaptive proton therapy, Med. Phys. 42, 4449{4459 (2015). S. Kida, T. Nakamoto, M. Nakano, K. Nawa, A. Haga, J. Kotoku, H. Yamashita, and K. Nakagawa, Cone Beam Computed Tomography Image Quality Improvement Using a Deep Convolutional Neural Network., Cureus 10, e2548 (2018). S. Xie, C. Yang, Z. Zhang, and H. Li, Scatter Artifacts Removal Using Learning-Based Method for CBCT in IGRT System, IEEE Access 6, 78031{78037 (2018). J. Maier, E. Eulig, T. V oth, M. Knaup, J. Kuntz, S. Sawall, and M. Kachelrie, Real- time scatter estimation for medical CT using the deep scatter estimation: Method and robustness analysis with respect to di erent anatomies, dose levels, tube voltages, and data truncation, Med. Phys. 46, 238{249 (2019). D. C. Hansen, G. Landry, F. Kamp, M. Li, C. Belka, K. Parodi, and C. Kurz, ScatterNet: A convolutional neural network for conebeam CT intensity correction, Med. Phys. 45, 4916{4926 (2018). X. Liang, L. Chen, D. Nguyen, Z. Zhou, X. Gu, M. Yang, J. Wang, and S. Jiang, Gen- erating synthesized computed tomography (CT) from cone-beam computed tomography (CBCT) using CycleGAN for adaptive radiation therapy, Phys. Med. Biol. 64, 125002 (2019). C. Kurz, M. Maspero, M. H. Savenije, G. Landry, F. Kamp, M. Pinto, M. Li, K. Parodi, C. Belka, and C. A. van den Berg, CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation, Phys. Med. Biol. 64, 225004 (2019). CBCT-to-CT synthesis with a single neural network page 21 P. Meyer, V. Noblet, C. Mazzara, and A. Lallement, Survey on deep learning for radio- therapy, Comput. Biol. Med. 98, 126{146 (2018). B. Sahiner, A. Pezeshk, L. M. Hadjiiski, X. Wang, K. Drukker, K. H. Cha, R. M. Summers, and M. L. Giger, Deep learning in medical imaging and radiation therapy, Med. Phys. 46, e1{e36 (2019). X. Han, MR-based synthetic CT generation using a deep convolutional neural network method, Med. Phys. 44, 1408{1419 (2017). M. Maspero, M. H. F. Savenije, A. M. Dinkla, P. R. Seevinck, M. P. W. Intven, I. M. Jurgenliemk-Schulz, L. G. W. Kerkmeijer, and C. A. T. van den Berg, Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy, Phys. Med. Biol. 63, 185001 (2018). J. Harms, Y. Lei, T. Wang, R. Zhang, J. Zhou, X. Tang, W. J. Curran, T. Liu, and X. Yang, Paired cycle-GAN based image correction for quantitative cone-beam CT, Med. Phys. (2019). H. C. de Boer and B. J. Heijmen, A protocol for the reduction of systematic patient setup errors with minimal portal imaging workload, Int. J Radiat. Oncol. Biol. Phys. 50, 1350{1365 (2001). G. Borgefors, Hierarchical chamfer matching: a parametric edge matching algorithm, IEEE Transactions on Pattern Analysis and Machine Intelligence 10, 849{865 (1988). D. H. Hristov and B. G. Fallone, A grey-level image alignment algorithm for registration of portal images and digitally reconstructed radiographs, Med. Phys. 23, 75{84 (1996). A. Roche, G. Malandain, X. Pennec, and N. Ayache, The correlation ratio as a new similarity measure for multimodal image registration, in Lecture Notes in Computer Science, MICCAI, pages 1115{1124, Springer, Berlin, Heidelberg, 1998. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in Proceedings of the IEEE international conference on computer vision, pages 2223{2232, 2017. Last edited: December 25, 2019 page 22 Maspero M J. M. Wolterink, A. M. Dinkla, M. H. F. Savenije, P. R. Seevinck, C. A. T. van den Berg, and I. I sgum, Deep MR to CT Synthesis Using Unpaired Data, in Simulation and Synthesis in Medical Imaging, edited by S. A. Tsaftaris, A. Gooya, A. F. Frangi, and J. L. Prince, pages 14{23, Cham, 2017, Springer International Publishing. J. Johnson, A. Alahi, and L. Fei-Fei, Perceptual Losses for Real-Time Style Transfer and Super-Resolution, Computer Vision ECCV 2016 , 694{711 (2016). P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, Image-to-Image Translation with Condi- tional Adversarial Networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5967{5976, IEEE, 2017. D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, arXiv preprint arXiv:1412.6980 (2014). D. Ulyanov, A. Vedaldi, and V. Lempitsky, Instance Normalization: The Missing Ingre- dient for Fast Stylization, arXiv preprint arXiv:1607.08022 (2016). I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, Generative Adversarial Nets, in Advances in Neural Information Processing Systems 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, pages 2672{2680, Curran Associates, Inc., 2014. D. A. Low, Gamma Dose Distribution Evaluation Tool, J. Phys. 250, 012071 (2010). M. Maspero, M. D. Tyyger, R. H. N. Tijssen, P. R. Seevinck, M. P. W. Intven, and C. A. T. van den Berg, Feasibility of magnetic resonance imaging-only rectum radio- therapy with a commercial synthetic computed tomography generation solution, Phys. Imag. Radiat. Oncol.y 7, 58{64 (2018). E. van Lin, L. van der Vight, H. Huizenga, J. Kaanders, and A. Visser, Set-up improve- ment in head and neck radiotherapy using a 3D o -line EPID-based correction protocol and a customised head and neck support, Radiother. Oncol. 68, 137{148 (2003). G. Landry, D. Hansen, F. Kamp, M. Li, B. Hoyle, J. Weller, K. Parodi, C. Belka, and C. Kurz, Comparing Unet training with three di erent datasets to correct CBCT images for prostate radiotherapy dose calculations, Phys. Med. Biol. 64, 035011 (2019). CBCT-to-CT synthesis with a single neural network page 23 Y. Li, J. Zhu, Z. Liu, J. Teng, Q. Xie, L. Zhang, X. Liu, J. Shi, and L. Chen, A preliminary study of using a deep convolution neural network to generate synthesized CT images based on CBCT for adaptive radiotherapy of nasopharyngeal carcinoma, Phys. Med. Biol. 64 (2019). C. Paganelli, G. Meschini, S. Molinelli, M. Riboldi, and G. Baroni, Patient-speci c validation of deformable image registration in radiation therapy: Overview and caveats, Med. Phys. 45, e908{e922 (2018). Last edited: December 25, 2019

Journal

Electrical Engineering and Systems SciencearXiv (Cornell University)

Published: Dec 23, 2019

References