2024
Montin, Eros; Deniz, Cem M.; Kijowski, Richard; Youm, Thomas; Lattanzi, Riccardo
In: Informatics in Medicine Unlocked, vol. 45, pp. 101444, 2024, ISSN: 2352-9148.
@article{MONTIN2024101444,
title = {The impact of data augmentation and transfer learning on the performance of deep learning models for the segmentation of the hip on 3D magnetic resonance images},
author = {Eros Montin and Cem M. Deniz and Richard Kijowski and Thomas Youm and Riccardo Lattanzi},
url = {https://www.sciencedirect.com/science/article/pii/S2352914823002903},
doi = {https://doi.org/10.1016/j.imu.2023.101444},
issn = {2352-9148},
year = {2024},
date = {2024-01-01},
journal = {Informatics in Medicine Unlocked},
volume = {45},
pages = {101444},
abstract = {Different pathologies of the hip are characterized by the abnormal shape of the bony structures of the joint, namely the femur and the acetabulum. Three-dimensional (3D) models of the hip can be used for diagnosis, biomechanical simulation, and planning of surgical treatments. These models can be generated by building 3D surfaces of the joint's structures segmented on magnetic resonance (MR) images. Deep learning can avoid time-consuming manual segmentations, but its performance depends on the amount and quality of the available training data. Data augmentation and transfer learning are two approaches used when there is only a limited number of datasets. In particular, data augmentation can be used to artificially increase the size and diversity of the training datasets, whereas transfer learning can be used to build the desired model on top of a model previously trained with similar data. This study investigates the effect of data augmentation and transfer learning on the performance of deep learning for the automatic segmentation of the femur and acetabulum on 3D MR images of patients diagnosed with femoroacetabular impingement. Transfer learning was applied starting from a model trained for the segmentation of the bony structures of the shoulder joint, which bears some resemblance to the hip joint. Our results suggest that data augmentation is more effective than transfer learning, yielding a Dice similarity coefficient compared to ground-truth manual segmentations of 0.84 and 0.89 for the acetabulum and femur, respectively, whereas the Dice coefficient was 0.78 and 0.88 for the model based on transfer learning. The Accuracy for the two anatomical regions was 0.95 and 0.97 when using data augmentation, and 0.87 and 0.96 when using transfer learning. Data augmentation can improve the performance of deep learning models by increasing the diversity of the training dataset and making the models more robust to noise and variations in image quality. The proposed segmentation model could be combined with radiomic analysis for the automatic evaluation of hip pathologies.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Different pathologies of the hip are characterized by the abnormal shape of the bony structures of the joint, namely the femur and the acetabulum. Three-dimensional (3D) models of the hip can be used for diagnosis, biomechanical simulation, and planning of surgical treatments. These models can be generated by building 3D surfaces of the joint's structures segmented on magnetic resonance (MR) images. Deep learning can avoid time-consuming manual segmentations, but its performance depends on the amount and quality of the available training data. Data augmentation and transfer learning are two approaches used when there is only a limited number of datasets. In particular, data augmentation can be used to artificially increase the size and diversity of the training datasets, whereas transfer learning can be used to build the desired model on top of a model previously trained with similar data. This study investigates the effect of data augmentation and transfer learning on the performance of deep learning for the automatic segmentation of the femur and acetabulum on 3D MR images of patients diagnosed with femoroacetabular impingement. Transfer learning was applied starting from a model trained for the segmentation of the bony structures of the shoulder joint, which bears some resemblance to the hip joint. Our results suggest that data augmentation is more effective than transfer learning, yielding a Dice similarity coefficient compared to ground-truth manual segmentations of 0.84 and 0.89 for the acetabulum and femur, respectively, whereas the Dice coefficient was 0.78 and 0.88 for the model based on transfer learning. The Accuracy for the two anatomical regions was 0.95 and 0.97 when using data augmentation, and 0.87 and 0.96 when using transfer learning. Data augmentation can improve the performance of deep learning models by increasing the diversity of the training dataset and making the models more robust to noise and variations in image quality. The proposed segmentation model could be combined with radiomic analysis for the automatic evaluation of hip pathologies.
2023
Rajamohan, H. R.; Wang, T.; Leung, K.; Chang, G.; Cho, K.; Kijowski, R.; Deniz, C. M.
Prediction of total knee replacement using deep learning analysis of knee MRI Journal Article
In: Scientific Reports, vol. 13, iss. 1, no. 6922, 2023.
@article{Rajamohan2023,
title = {Prediction of total knee replacement using deep learning analysis of knee MRI},
author = {H.R. Rajamohan and T. Wang and K. Leung and G. Chang and K. Cho and R. Kijowski and C.M. Deniz},
doi = {10.1038/s41598-023-33934-1},
year = {2023},
date = {2023-04-28},
urldate = {2023-04-28},
journal = {Scientific Reports},
volume = {13},
number = {6922},
issue = {1},
abstract = {Current methods for assessing knee osteoarthritis (OA) do not provide comprehensive information to make robust and accurate outcome predictions. Deep learning (DL) risk assessment models were developed to predict the progression of knee OA to total knee replacement (TKR) over a 108-month follow-up period using baseline knee MRI. Participants of our retrospective study consisted of 353 case-control pairs of subjects from the Osteoarthritis Initiative with and without TKR over a 108-month follow-up period matched according to age, sex, ethnicity, and body mass index. A traditional risk assessment model was created to predict TKR using baseline clinical risk factors. DL models were created to predict TKR using baseline knee radiographs and MRI. All DL models had significantly higher (p < 0.001) AUCs than the traditional model. The MRI and radiograph ensemble model and MRI ensemble model (where TKR risk predicted by several contrast-specific DL models were averaged to get the ensemble TKR risk prediction) had the highest AUCs of 0.90 (80% sensitivity and 85% specificity) and 0.89 (79% sensitivity and 86% specificity), respectively, which were significantly higher (p < 0.05) than the AUCs of the radiograph and multiple MRI models (where the DL models were trained to predict TKR risk using single contrast or 2 contrasts together as input). DL models using baseline MRI had a higher diagnostic performance for predicting TKR than a traditional model using baseline clinical risk factors and a DL model using baseline knee radiographs.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Current methods for assessing knee osteoarthritis (OA) do not provide comprehensive information to make robust and accurate outcome predictions. Deep learning (DL) risk assessment models were developed to predict the progression of knee OA to total knee replacement (TKR) over a 108-month follow-up period using baseline knee MRI. Participants of our retrospective study consisted of 353 case-control pairs of subjects from the Osteoarthritis Initiative with and without TKR over a 108-month follow-up period matched according to age, sex, ethnicity, and body mass index. A traditional risk assessment model was created to predict TKR using baseline clinical risk factors. DL models were created to predict TKR using baseline knee radiographs and MRI. All DL models had significantly higher (p < 0.001) AUCs than the traditional model. The MRI and radiograph ensemble model and MRI ensemble model (where TKR risk predicted by several contrast-specific DL models were averaged to get the ensemble TKR risk prediction) had the highest AUCs of 0.90 (80% sensitivity and 85% specificity) and 0.89 (79% sensitivity and 86% specificity), respectively, which were significantly higher (p < 0.05) than the AUCs of the radiograph and multiple MRI models (where the DL models were trained to predict TKR risk using single contrast or 2 contrasts together as input). DL models using baseline MRI had a higher diagnostic performance for predicting TKR than a traditional model using baseline clinical risk factors and a DL model using baseline knee radiographs.
Kijowski, Richard; Fritz, Jan; Deniz, Cem M.
Deep learning applications in osteoarthritis imaging Journal Article
In: Skeletal Radiol, vol. 52, no. 11, pp. 2225–2238, 2023, ISSN: 1432-2161.
@article{Kijowski2023,
title = {Deep learning applications in osteoarthritis imaging},
author = {Richard Kijowski and Jan Fritz and Cem M. Deniz},
doi = {10.1007/s00256-023-04296-6},
issn = {1432-2161},
year = {2023},
date = {2023-02-09},
journal = {Skeletal Radiol},
volume = {52},
number = {11},
pages = {2225--2238},
publisher = {Springer Science and Business Media LLC},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Cigdem, Ozkan; Deniz, Cem M
Artificial intelligence in knee osteoarthritis: A comprehensive review for 2022 Journal Article
In: Osteoarthritis Imaging, vol. 3, no. 3, pp. 100161, 2023, ISSN: 2772-6541.
@article{CIGDEM2023100161,
title = {Artificial intelligence in knee osteoarthritis: A comprehensive review for 2022},
author = {Ozkan Cigdem and Cem M Deniz},
url = {https://www.sciencedirect.com/science/article/pii/S277265412300079X},
doi = {https://doi.org/10.1016/j.ostima.2023.100161},
issn = {2772-6541},
year = {2023},
date = {2023-01-01},
journal = {Osteoarthritis Imaging},
volume = {3},
number = {3},
pages = {100161},
abstract = {Objective
The aim of this literature review is to yield a comprehensive and exhaustive overview of the existing evidence and up-to-date applications of artificial intelligence for knee osteoarthritis.
Methods
A literature review was performed by using PubMed, Google Scholar, and IEEE databases for articles published in peer-reviewed journals in 2022. The articles focusing on the use of artificial intelligence in diagnosis and prognosis of knee osteoarthritis and accelerating the image acquisition were selected. For each selected study, the code availability, considered number of patients and knees, imaging type, covariates, grading type of osteoarthritis, models, validation approaches, objectives, and results were reviewed.
Results
395 articles were screened, and 35 of them were reviewed. Eight articles were based on diagnosis, six on prognosis prediction, three on classification, three on accelerated image acquisition, and 15 on segmentation of knee osteoarthritis. 57% of the articles used MRI, 26% radiography, 6% MRI together with radiography, 6% ultrasonography, and 6% only clinical data. 23% of the articles made the computer codes available for their study, and 26% used clinical data. External validation and nested cross-validation were used in 17% and 14% of articles, respectively.
Conclusions
The use of artificial intelligence provided a promising potential to enhance the detection and management of knee osteoarthritis. Translating the developed models into clinics is still in the early stages of development. The translation of artificial intelligence models is expected to be further examined in prospective studies to support clinicians in improving routine healthcare practice.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Objective
The aim of this literature review is to yield a comprehensive and exhaustive overview of the existing evidence and up-to-date applications of artificial intelligence for knee osteoarthritis.
Methods
A literature review was performed by using PubMed, Google Scholar, and IEEE databases for articles published in peer-reviewed journals in 2022. The articles focusing on the use of artificial intelligence in diagnosis and prognosis of knee osteoarthritis and accelerating the image acquisition were selected. For each selected study, the code availability, considered number of patients and knees, imaging type, covariates, grading type of osteoarthritis, models, validation approaches, objectives, and results were reviewed.
Results
395 articles were screened, and 35 of them were reviewed. Eight articles were based on diagnosis, six on prognosis prediction, three on classification, three on accelerated image acquisition, and 15 on segmentation of knee osteoarthritis. 57% of the articles used MRI, 26% radiography, 6% MRI together with radiography, 6% ultrasonography, and 6% only clinical data. 23% of the articles made the computer codes available for their study, and 26% used clinical data. External validation and nested cross-validation were used in 17% and 14% of articles, respectively.
Conclusions
The use of artificial intelligence provided a promising potential to enhance the detection and management of knee osteoarthritis. Translating the developed models into clinics is still in the early stages of development. The translation of artificial intelligence models is expected to be further examined in prospective studies to support clinicians in improving routine healthcare practice.
The aim of this literature review is to yield a comprehensive and exhaustive overview of the existing evidence and up-to-date applications of artificial intelligence for knee osteoarthritis.
Methods
A literature review was performed by using PubMed, Google Scholar, and IEEE databases for articles published in peer-reviewed journals in 2022. The articles focusing on the use of artificial intelligence in diagnosis and prognosis of knee osteoarthritis and accelerating the image acquisition were selected. For each selected study, the code availability, considered number of patients and knees, imaging type, covariates, grading type of osteoarthritis, models, validation approaches, objectives, and results were reviewed.
Results
395 articles were screened, and 35 of them were reviewed. Eight articles were based on diagnosis, six on prognosis prediction, three on classification, three on accelerated image acquisition, and 15 on segmentation of knee osteoarthritis. 57% of the articles used MRI, 26% radiography, 6% MRI together with radiography, 6% ultrasonography, and 6% only clinical data. 23% of the articles made the computer codes available for their study, and 26% used clinical data. External validation and nested cross-validation were used in 17% and 14% of articles, respectively.
Conclusions
The use of artificial intelligence provided a promising potential to enhance the detection and management of knee osteoarthritis. Translating the developed models into clinics is still in the early stages of development. The translation of artificial intelligence models is expected to be further examined in prospective studies to support clinicians in improving routine healthcare practice.
Hirvasniemi, J.; Runhaar, J.; Heijden, R. A.; Zokaeinikoo, M.; Yang, M.; Li, X.; Tan, J.; Rajamohan, H. R.; Zhou, Y.; Deniz, C. M.; Caliva, F.; Iriondo, C.; Lee, J. J.; Liu, F.; Martinez, A. M.; Namiri, N.; Pedoia, V.; Panfilov, E.; Bayramoglu, N.; Nguyen, H. H.; Nieminen, M. T.; Saarakkala, S.; Tiulpin, A.; Lin, E.; Li, A.; Li, V.; Dam, E. B.; Chaudhari, A. S.; Kijowski, R.; Bierma-Zeinstra, S.; Oei, E. H. G.; Klein, S.
In: Osteoarthritis and Cartilage, vol. 31, no. 1, pp. 115-125, 2023, ISSN: 1063-4584.
@article{HIRVASNIEMI2023115,
title = {The KNee OsteoArthritis Prediction (KNOAP2020) challenge: An image analysis challenge to predict incident symptomatic radiographic knee osteoarthritis from MRI and X-ray images},
author = {J. Hirvasniemi and J. Runhaar and R. A. Heijden and M. Zokaeinikoo and M. Yang and X. Li and J. Tan and H. R. Rajamohan and Y. Zhou and C. M. Deniz and F. Caliva and C. Iriondo and J. J. Lee and F. Liu and A. M. Martinez and N. Namiri and V. Pedoia and E. Panfilov and N. Bayramoglu and H. H. Nguyen and M. T. Nieminen and S. Saarakkala and A. Tiulpin and E. Lin and A. Li and V. Li and E. B. Dam and A. S. Chaudhari and R. Kijowski and S. Bierma-Zeinstra and E. H. G. Oei and S. Klein},
url = {https://www.sciencedirect.com/science/article/pii/S1063458422008640},
doi = {https://doi.org/10.1016/j.joca.2022.10.001},
issn = {1063-4584},
year = {2023},
date = {2023-01-01},
journal = {Osteoarthritis and Cartilage},
volume = {31},
number = {1},
pages = {115-125},
abstract = {Summary
Objectives
The KNee OsteoArthritis Prediction (KNOAP2020) challenge was organized to objectively compare methods for the prediction of incident symptomatic radiographic knee osteoarthritis within 78 months on a test set with blinded ground truth.
Design
The challenge participants were free to use any available data sources to train their models. A test set of 423 knees from the Prevention of Knee Osteoarthritis in Overweight Females (PROOF) study consisting of magnetic resonance imaging (MRI) and X-ray image data along with clinical risk factors at baseline was made available to all challenge participants. The ground truth outcomes, i.e., which knees developed incident symptomatic radiographic knee osteoarthritis (according to the combined ACR criteria) within 78 months, were not provided to the participants. To assess the performance of the submitted models, we used the area under the receiver operating characteristic curve (ROCAUC) and balanced accuracy (BACC).
Results
Seven teams submitted 23 entries in total. A majority of the algorithms were trained on data from the Osteoarthritis Initiative. The model with the highest ROCAUC (0.64 (95% confidence interval (CI): 0.57–0.70)) used deep learning to extract information from X-ray images combined with clinical variables. The model with the highest BACC (0.59 (95% CI: 0.52–0.65)) ensembled three different models that used automatically extracted X-ray and MRI features along with clinical variables.
Conclusion
The KNOAP2020 challenge established a benchmark for predicting incident symptomatic radiographic knee osteoarthritis. Accurate prediction of incident symptomatic radiographic knee osteoarthritis is a complex and still unsolved problem requiring additional investigation.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Summary
Objectives
The KNee OsteoArthritis Prediction (KNOAP2020) challenge was organized to objectively compare methods for the prediction of incident symptomatic radiographic knee osteoarthritis within 78 months on a test set with blinded ground truth.
Design
The challenge participants were free to use any available data sources to train their models. A test set of 423 knees from the Prevention of Knee Osteoarthritis in Overweight Females (PROOF) study consisting of magnetic resonance imaging (MRI) and X-ray image data along with clinical risk factors at baseline was made available to all challenge participants. The ground truth outcomes, i.e., which knees developed incident symptomatic radiographic knee osteoarthritis (according to the combined ACR criteria) within 78 months, were not provided to the participants. To assess the performance of the submitted models, we used the area under the receiver operating characteristic curve (ROCAUC) and balanced accuracy (BACC).
Results
Seven teams submitted 23 entries in total. A majority of the algorithms were trained on data from the Osteoarthritis Initiative. The model with the highest ROCAUC (0.64 (95% confidence interval (CI): 0.57–0.70)) used deep learning to extract information from X-ray images combined with clinical variables. The model with the highest BACC (0.59 (95% CI: 0.52–0.65)) ensembled three different models that used automatically extracted X-ray and MRI features along with clinical variables.
Conclusion
The KNOAP2020 challenge established a benchmark for predicting incident symptomatic radiographic knee osteoarthritis. Accurate prediction of incident symptomatic radiographic knee osteoarthritis is a complex and still unsolved problem requiring additional investigation.
Objectives
The KNee OsteoArthritis Prediction (KNOAP2020) challenge was organized to objectively compare methods for the prediction of incident symptomatic radiographic knee osteoarthritis within 78 months on a test set with blinded ground truth.
Design
The challenge participants were free to use any available data sources to train their models. A test set of 423 knees from the Prevention of Knee Osteoarthritis in Overweight Females (PROOF) study consisting of magnetic resonance imaging (MRI) and X-ray image data along with clinical risk factors at baseline was made available to all challenge participants. The ground truth outcomes, i.e., which knees developed incident symptomatic radiographic knee osteoarthritis (according to the combined ACR criteria) within 78 months, were not provided to the participants. To assess the performance of the submitted models, we used the area under the receiver operating characteristic curve (ROCAUC) and balanced accuracy (BACC).
Results
Seven teams submitted 23 entries in total. A majority of the algorithms were trained on data from the Osteoarthritis Initiative. The model with the highest ROCAUC (0.64 (95% confidence interval (CI): 0.57–0.70)) used deep learning to extract information from X-ray images combined with clinical variables. The model with the highest BACC (0.59 (95% CI: 0.52–0.65)) ensembled three different models that used automatically extracted X-ray and MRI features along with clinical variables.
Conclusion
The KNOAP2020 challenge established a benchmark for predicting incident symptomatic radiographic knee osteoarthritis. Accurate prediction of incident symptomatic radiographic knee osteoarthritis is a complex and still unsolved problem requiring additional investigation.
Dam, Erik B; Desai, Arjun D; Deniz, Cem M; Rajamohan, Haresh R; Regatte, Ravinder; Iriondo, Claudia; Pedoia, Valentina; Majumdar, Sharmila; Perslev, Mathias; Igel, Christian; Pai, Akshay; Gaj, Sibaji; Yang, Mingrui; Nakamura, Kunio; Li, Xiaojuan; Maqbool, Hasan; Irmakci, Ismail; Song, Sang-Eun; Bagci, Ulas; Hargreaves, Brian; Gold, Garry; Chaudhari, Akshay
Towards automatic cartilage quantification in clinical trials – Continuing from the 2019 IWOAI knee segmentation challenge Journal Article
In: Osteoarthritis Imaging, vol. 3, no. 1, pp. 100087, 2023, ISSN: 2772-6541.
@article{DAM2023100087,
title = {Towards automatic cartilage quantification in clinical trials – Continuing from the 2019 IWOAI knee segmentation challenge},
author = {Erik B Dam and Arjun D Desai and Cem M Deniz and Haresh R Rajamohan and Ravinder Regatte and Claudia Iriondo and Valentina Pedoia and Sharmila Majumdar and Mathias Perslev and Christian Igel and Akshay Pai and Sibaji Gaj and Mingrui Yang and Kunio Nakamura and Xiaojuan Li and Hasan Maqbool and Ismail Irmakci and Sang-Eun Song and Ulas Bagci and Brian Hargreaves and Garry Gold and Akshay Chaudhari},
url = {https://www.sciencedirect.com/science/article/pii/S2772654123000028},
doi = {https://doi.org/10.1016/j.ostima.2023.100087},
issn = {2772-6541},
year = {2023},
date = {2023-01-01},
journal = {Osteoarthritis Imaging},
volume = {3},
number = {1},
pages = {100087},
abstract = {Objective
To evaluate whether the deep learning (DL) segmentation methods from the six teams that participated in the IWOAI 2019 Knee Cartilage Segmentation Challenge are appropriate for quantifying cartilage loss in longitudinal clinical trials.
Design
We included 556 subjects from the Osteoarthritis Initiative study with manually read cartilage volume scores for the baseline and 1-year visits. The teams used their methods originally trained for the IWOAI 2019 challenge to segment the 1130 knee MRIs. These scans were anonymized and the teams were blinded to any subject or visit identifiers. Two teams also submitted updated methods. The resulting 9,040 segmentations are available online. The segmentations included tibial, femoral, and patellar compartments. In post-processing, we extracted medial and lateral tibial compartments and geometrically defined central medial and lateral femoral sub-compartments. The primary study outcome was the sensitivity to measure cartilage loss as defined by the standardized response mean (SRM).
Results
For the tibial compartments, several of the DL segmentation methods had SRMs similar to the gold standard manual method. The highest DL SRM was for the lateral tibial compartment at 0.38 (the gold standard had 0.34). For the femoral compartments, the gold standard had higher SRMs than the automatic methods at 0.31/0.30 for medial/lateral compartments.
Conclusion
The lower SRMs for the DL methods in the femoral compartments at 0.2 were possibly due to the simple sub-compartment extraction done during post-processing. The study demonstrated that state-of-the-art DL segmentation methods may be used in standardized longitudinal single-scanner clinical trials for well-defined cartilage compartments.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Objective
To evaluate whether the deep learning (DL) segmentation methods from the six teams that participated in the IWOAI 2019 Knee Cartilage Segmentation Challenge are appropriate for quantifying cartilage loss in longitudinal clinical trials.
Design
We included 556 subjects from the Osteoarthritis Initiative study with manually read cartilage volume scores for the baseline and 1-year visits. The teams used their methods originally trained for the IWOAI 2019 challenge to segment the 1130 knee MRIs. These scans were anonymized and the teams were blinded to any subject or visit identifiers. Two teams also submitted updated methods. The resulting 9,040 segmentations are available online. The segmentations included tibial, femoral, and patellar compartments. In post-processing, we extracted medial and lateral tibial compartments and geometrically defined central medial and lateral femoral sub-compartments. The primary study outcome was the sensitivity to measure cartilage loss as defined by the standardized response mean (SRM).
Results
For the tibial compartments, several of the DL segmentation methods had SRMs similar to the gold standard manual method. The highest DL SRM was for the lateral tibial compartment at 0.38 (the gold standard had 0.34). For the femoral compartments, the gold standard had higher SRMs than the automatic methods at 0.31/0.30 for medial/lateral compartments.
Conclusion
The lower SRMs for the DL methods in the femoral compartments at 0.2 were possibly due to the simple sub-compartment extraction done during post-processing. The study demonstrated that state-of-the-art DL segmentation methods may be used in standardized longitudinal single-scanner clinical trials for well-defined cartilage compartments.
To evaluate whether the deep learning (DL) segmentation methods from the six teams that participated in the IWOAI 2019 Knee Cartilage Segmentation Challenge are appropriate for quantifying cartilage loss in longitudinal clinical trials.
Design
We included 556 subjects from the Osteoarthritis Initiative study with manually read cartilage volume scores for the baseline and 1-year visits. The teams used their methods originally trained for the IWOAI 2019 challenge to segment the 1130 knee MRIs. These scans were anonymized and the teams were blinded to any subject or visit identifiers. Two teams also submitted updated methods. The resulting 9,040 segmentations are available online. The segmentations included tibial, femoral, and patellar compartments. In post-processing, we extracted medial and lateral tibial compartments and geometrically defined central medial and lateral femoral sub-compartments. The primary study outcome was the sensitivity to measure cartilage loss as defined by the standardized response mean (SRM).
Results
For the tibial compartments, several of the DL segmentation methods had SRMs similar to the gold standard manual method. The highest DL SRM was for the lateral tibial compartment at 0.38 (the gold standard had 0.34). For the femoral compartments, the gold standard had higher SRMs than the automatic methods at 0.31/0.30 for medial/lateral compartments.
Conclusion
The lower SRMs for the DL methods in the femoral compartments at 0.2 were possibly due to the simple sub-compartment extraction done during post-processing. The study demonstrated that state-of-the-art DL segmentation methods may be used in standardized longitudinal single-scanner clinical trials for well-defined cartilage compartments.
2021
Desai, Arjun D.; Caliva, Francesco; Iriondo, Claudia; Mortazi, Aliasghar; Jambawalikar, Sachin; Bagci, Ulas; Perslev, Mathias; Igel, Christian; Dam, Erik B.; Gaj, Sibaji; Yang, Mingrui; Li, Xiaojuan; Deniz, Cem M.; Juras, Vladimir; Regatte, Ravinder; Gold, Garry E.; Hargreaves, Brian A.; Pedoia, Valentina; Chaudhari, Akshay S.; Khosravan, Naji; Torigian, Drew; Ellermann, Jutta; Akcakaya, Mehmet; Tibrewala, Radhika; Flament, Io; O’Brien, Matthew; Majumdar, Sharmila; Nakamura, Kunio; Pai, Akshay
In: Radiology: Artificial Intelligence, vol. 3, no. 3, pp. e200078, 2021.
@article{doi:10.1148/ryai.2021200078,
title = {The International Workshop on Osteoarthritis Imaging Knee MRI Segmentation Challenge: A Multi-Institute Evaluation and Analysis Framework on a Standardized Dataset},
author = {Arjun D. Desai and Francesco Caliva and Claudia Iriondo and Aliasghar Mortazi and Sachin Jambawalikar and Ulas Bagci and Mathias Perslev and Christian Igel and Erik B. Dam and Sibaji Gaj and Mingrui Yang and Xiaojuan Li and Cem M. Deniz and Vladimir Juras and Ravinder Regatte and Garry E. Gold and Brian A. Hargreaves and Valentina Pedoia and Akshay S. Chaudhari and Naji Khosravan and Drew Torigian and Jutta Ellermann and Mehmet Akcakaya and Radhika Tibrewala and Io Flament and Matthew O’Brien and Sharmila Majumdar and Kunio Nakamura and Akshay Pai},
url = {https://doi.org/10.1148/ryai.2021200078},
doi = {10.1148/ryai.2021200078},
year = {2021},
date = {2021-01-01},
journal = {Radiology: Artificial Intelligence},
volume = {3},
number = {3},
pages = {e200078},
abstract = {Purpose To organize a multi-institute knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. Materials and Methods A dataset partition consisting of three-dimensional knee MRI from 88 retrospective patients at two time points (baseline and 1-year follow-up) with ground truth articular (femoral, tibial, and patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated against ground truth segmentations using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a holdout test set. Similarities in automated segmentations were measured using pairwise Dice coefficient correlations. Articular cartilage thickness was computed longitudinally and with scans. Correlation between thickness error and segmentation metrics was measured using the Pearson correlation coefficient. Two empirical upper bounds for ensemble performance were computed using combinations of model outputs that consolidated true positives and true negatives. Results Six teams (T1–T6) submitted entries for the challenge. No differences were observed across any segmentation metrics for any tissues (P = .99) among the four top-performing networks (T2, T3, T4, T6). Dice coefficient correlations between network pairs were high (> 0.85). Per-scan thickness errors were negligible among networks T1–T4 (P = .99), and longitudinal changes showed minimal bias (< 0.03 mm). Low correlations (ρ < 0.41) were observed between segmentation metrics and thickness error. The majority-vote ensemble was comparable to top-performing networks (P = .99). Empirical upper-bound performances were similar for both combinations (P = .99). Conclusion Diverse networks learned to segment the knee similarly, where high segmentation accuracy did not correlate with cartilage thickness accuracy and voting ensembles did not exceed individual network performance. See also the commentary by Elhalawani and Mak in this issue. Keywords: Cartilage, Knee, MR-Imaging, Segmentation © RSNA, 2020 Supplemental material is available for this article.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Purpose To organize a multi-institute knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. Materials and Methods A dataset partition consisting of three-dimensional knee MRI from 88 retrospective patients at two time points (baseline and 1-year follow-up) with ground truth articular (femoral, tibial, and patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated against ground truth segmentations using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a holdout test set. Similarities in automated segmentations were measured using pairwise Dice coefficient correlations. Articular cartilage thickness was computed longitudinally and with scans. Correlation between thickness error and segmentation metrics was measured using the Pearson correlation coefficient. Two empirical upper bounds for ensemble performance were computed using combinations of model outputs that consolidated true positives and true negatives. Results Six teams (T1–T6) submitted entries for the challenge. No differences were observed across any segmentation metrics for any tissues (P = .99) among the four top-performing networks (T2, T3, T4, T6). Dice coefficient correlations between network pairs were high (> 0.85). Per-scan thickness errors were negligible among networks T1–T4 (P = .99), and longitudinal changes showed minimal bias (< 0.03 mm). Low correlations (ρ < 0.41) were observed between segmentation metrics and thickness error. The majority-vote ensemble was comparable to top-performing networks (P = .99). Empirical upper-bound performances were similar for both combinations (P = .99). Conclusion Diverse networks learned to segment the knee similarly, where high segmentation accuracy did not correlate with cartilage thickness accuracy and voting ensembles did not exceed individual network performance. See also the commentary by Elhalawani and Mak in this issue. Keywords: Cartilage, Knee, MR-Imaging, Segmentation © RSNA, 2020 Supplemental material is available for this article.
2020
Rodrigues, Tatiane Cantarelli; Deniz, Cem M; Alaia, Erin F; Gorelik, Natalia; Babb, James S; Dublin, Jared; Gyftopoulos, Soterios
Three-dimensional MRI Bone Models of the Glenohumeral Joint Using Deep Learning: Evaluation of Normal Anatomy and Glenoid Bone Loss Journal Article
In: Radiology: Artificial Intelligence, vol. 2, no. 5, 2020.
@article{Rodrigues2020,
title = {Three-dimensional MRI Bone Models of the Glenohumeral Joint Using Deep Learning: Evaluation of Normal Anatomy and Glenoid Bone Loss},
author = {Tatiane Cantarelli Rodrigues and Cem M Deniz and Erin F Alaia and Natalia Gorelik and James S Babb and Jared Dublin and Soterios Gyftopoulos},
doi = {10.1148/ryai.2020190116},
year = {2020},
date = {2020-09-09},
journal = {Radiology: Artificial Intelligence},
volume = {2},
number = {5},
abstract = {Purpose: To use convolutional neural networks (CNNs) for fully automated MRI segmentation of the glenohumeral joint and evaluate the accuracy of three-dimensional (3D) MRI models created with this method.
Materials and methods: Shoulder MR images of 100 patients (average age, 44 years; range, 14-80 years; 60 men) were retrospectively collected from September 2013 to August 2018. CNNs were used to develop a fully automated segmentation model for proton density-weighted images. Shoulder MR images from an additional 50 patients (mean age, 33 years; range, 16-65 years; 35 men) were retrospectively collected from May 2014 to April 2019 to create 3D MRI glenohumeral models by transfer learning using Dixon-based sequences. Two musculoskeletal radiologists performed measurements on fully and semiautomated segmented 3D MRI models to assess glenohumeral anatomy, glenoid bone loss (GBL), and their impact on treatment selection. Performance of the CNNs was evaluated using Dice similarity coefficient (DSC), sensitivity, precision, and surface-based distance measurements. Measurements were compared using matched-pairs Wilcoxon signed rank test.
Results: The two-dimensional CNN model for the humerus and glenoid achieved a DSC of 0.95 and 0.86, a precision of 95.5% and 87.5%, an average precision of 98.6% and 92.3%, and a sensitivity of 94.8% and 86.1%, respectively. The 3D CNN model, for the humerus and glenoid, achieved a DSC of 0.95 and 0.86, precision of 95.1% and 87.1%, an average precision of 98.7% and 91.9%, and a sensitivity of 94.9% and 85.6%, respectively. There was no difference between glenoid and humeral head width fully and semiautomated 3D model measurements (P value range, .097-.99).
Conclusion: CNNs could potentially be used in clinical practice to provide rapid and accurate 3D MRI glenohumeral bone models and GBL measurements.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Purpose: To use convolutional neural networks (CNNs) for fully automated MRI segmentation of the glenohumeral joint and evaluate the accuracy of three-dimensional (3D) MRI models created with this method.
Materials and methods: Shoulder MR images of 100 patients (average age, 44 years; range, 14-80 years; 60 men) were retrospectively collected from September 2013 to August 2018. CNNs were used to develop a fully automated segmentation model for proton density-weighted images. Shoulder MR images from an additional 50 patients (mean age, 33 years; range, 16-65 years; 35 men) were retrospectively collected from May 2014 to April 2019 to create 3D MRI glenohumeral models by transfer learning using Dixon-based sequences. Two musculoskeletal radiologists performed measurements on fully and semiautomated segmented 3D MRI models to assess glenohumeral anatomy, glenoid bone loss (GBL), and their impact on treatment selection. Performance of the CNNs was evaluated using Dice similarity coefficient (DSC), sensitivity, precision, and surface-based distance measurements. Measurements were compared using matched-pairs Wilcoxon signed rank test.
Results: The two-dimensional CNN model for the humerus and glenoid achieved a DSC of 0.95 and 0.86, a precision of 95.5% and 87.5%, an average precision of 98.6% and 92.3%, and a sensitivity of 94.8% and 86.1%, respectively. The 3D CNN model, for the humerus and glenoid, achieved a DSC of 0.95 and 0.86, precision of 95.1% and 87.1%, an average precision of 98.7% and 91.9%, and a sensitivity of 94.9% and 85.6%, respectively. There was no difference between glenoid and humeral head width fully and semiautomated 3D model measurements (P value range, .097-.99).
Conclusion: CNNs could potentially be used in clinical practice to provide rapid and accurate 3D MRI glenohumeral bone models and GBL measurements.
Materials and methods: Shoulder MR images of 100 patients (average age, 44 years; range, 14-80 years; 60 men) were retrospectively collected from September 2013 to August 2018. CNNs were used to develop a fully automated segmentation model for proton density-weighted images. Shoulder MR images from an additional 50 patients (mean age, 33 years; range, 16-65 years; 35 men) were retrospectively collected from May 2014 to April 2019 to create 3D MRI glenohumeral models by transfer learning using Dixon-based sequences. Two musculoskeletal radiologists performed measurements on fully and semiautomated segmented 3D MRI models to assess glenohumeral anatomy, glenoid bone loss (GBL), and their impact on treatment selection. Performance of the CNNs was evaluated using Dice similarity coefficient (DSC), sensitivity, precision, and surface-based distance measurements. Measurements were compared using matched-pairs Wilcoxon signed rank test.
Results: The two-dimensional CNN model for the humerus and glenoid achieved a DSC of 0.95 and 0.86, a precision of 95.5% and 87.5%, an average precision of 98.6% and 92.3%, and a sensitivity of 94.8% and 86.1%, respectively. The 3D CNN model, for the humerus and glenoid, achieved a DSC of 0.95 and 0.86, precision of 95.1% and 87.1%, an average precision of 98.7% and 91.9%, and a sensitivity of 94.9% and 85.6%, respectively. There was no difference between glenoid and humeral head width fully and semiautomated 3D model measurements (P value range, .097-.99).
Conclusion: CNNs could potentially be used in clinical practice to provide rapid and accurate 3D MRI glenohumeral bone models and GBL measurements.
Leung, Kevin; Zhang, Bofei; Tan, Jimin; Shen, Yiqiu; Geras, Krzysztof; Babb, James S; Cho, Kyunghyun; Chang, Gregory; Deniz, Cem M
In: Radiology, vol. 296, no. 3, pp. 584-593, 2020.
@article{Leung2020,
title = {Prediction of Total Knee Replacement and Diagnosis of Osteoarthritis by Using Deep Learning on Knee Radiographs: Data from the Osteoarthritis Initiative},
author = {Kevin Leung and Bofei Zhang and Jimin Tan and Yiqiu Shen and Krzysztof Geras and James S Babb and Kyunghyun Cho and Gregory Chang and Cem M Deniz},
doi = {10.1148/radiol.2020192091},
year = {2020},
date = {2020-06-23},
journal = {Radiology},
volume = {296},
number = {3},
pages = {584-593},
abstract = {Background The methods for assessing knee osteoarthritis (OA) do not provide enough comprehensive information to make robust and accurate outcome predictions. Purpose To develop a deep learning (DL) prediction model for risk of OA progression by using knee radiographs in patients who underwent total knee replacement (TKR) and matched control patients who did not undergo TKR. Materials and Methods In this retrospective analysis that used data from the OA Initiative, a DL model on knee radiographs was developed to predict both the likelihood of a patient undergoing TKR within 9 years and Kellgren-Lawrence (KL) grade. Study participants included a case-control matched subcohort between 45 and 79 years. Patients were matched to control patients according to age, sex, ethnicity, and body mass index. The proposed model used a transfer learning approach based on the ResNet34 architecture with sevenfold nested cross-validation. Receiver operating characteristic curve analysis and conditional logistic regression assessed model performance for predicting probability and risk of TKR compared with clinical observations and two binary outcome prediction models on the basis of radiographic readings: KL grade and OA Research Society International (OARSI) grade. Results Evaluated were 728 participants including 324 patients (mean age, 64 years ± 8 [standard deviation]; 222 women) and 324 control patients (mean age, 64 years ± 8; 222 women). The prediction model based on DL achieved an area under the receiver operating characteristic curve (AUC) of 0.87 (95% confidence interval [CI]: 0.85, 0.90), outperforming a baseline prediction model by using KL grade with an AUC of 0.74 (95% CI: 0.71, 0.77; P < .001). The risk for TKR increased with probability that a person will undergo TKR from the DL model (odds ratio [OR], 7.7; 95% CI: 2.3, 25; P < .001), KL grade (OR, 1.92; 95% CI: 1.17, 3.13; P = .009), and OARSI grade (OR, 1.20; 95% CI: 0.41, 3.50; P = .73). Conclusion The proposed deep learning model better predicted risk of total knee replacement in osteoarthritis than did binary outcome models by using standard grading systems. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Richardson in this issue. },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background The methods for assessing knee osteoarthritis (OA) do not provide enough comprehensive information to make robust and accurate outcome predictions. Purpose To develop a deep learning (DL) prediction model for risk of OA progression by using knee radiographs in patients who underwent total knee replacement (TKR) and matched control patients who did not undergo TKR. Materials and Methods In this retrospective analysis that used data from the OA Initiative, a DL model on knee radiographs was developed to predict both the likelihood of a patient undergoing TKR within 9 years and Kellgren-Lawrence (KL) grade. Study participants included a case-control matched subcohort between 45 and 79 years. Patients were matched to control patients according to age, sex, ethnicity, and body mass index. The proposed model used a transfer learning approach based on the ResNet34 architecture with sevenfold nested cross-validation. Receiver operating characteristic curve analysis and conditional logistic regression assessed model performance for predicting probability and risk of TKR compared with clinical observations and two binary outcome prediction models on the basis of radiographic readings: KL grade and OA Research Society International (OARSI) grade. Results Evaluated were 728 participants including 324 patients (mean age, 64 years ± 8 [standard deviation]; 222 women) and 324 control patients (mean age, 64 years ± 8; 222 women). The prediction model based on DL achieved an area under the receiver operating characteristic curve (AUC) of 0.87 (95% confidence interval [CI]: 0.85, 0.90), outperforming a baseline prediction model by using KL grade with an AUC of 0.74 (95% CI: 0.71, 0.77; P < .001). The risk for TKR increased with probability that a person will undergo TKR from the DL model (odds ratio [OR], 7.7; 95% CI: 2.3, 25; P < .001), KL grade (OR, 1.92; 95% CI: 1.17, 3.13; P = .009), and OARSI grade (OR, 1.20; 95% CI: 0.41, 3.50; P = .73). Conclusion The proposed deep learning model better predicted risk of total knee replacement in osteoarthritis than did binary outcome models by using standard grading systems. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Richardson in this issue.
Kopanoglu, Emre; Deniz, Cem M.; Erturk, M. Arcan; Wise, Richard G.
Specific absorption rate implications of within-scan patient head motion for ultra-high field MRI Journal Article
In: Magnetic Resonance in Medicine, vol. 84, pp. 2724– 2738, 2020.
@article{Kopanoglu2020,
title = {Specific absorption rate implications of within-scan patient head motion for ultra-high field MRI},
author = {Emre Kopanoglu and Cem M. Deniz and M. Arcan Erturk and Richard G. Wise},
doi = {10.1002/mrm.28276},
year = {2020},
date = {2020-04-17},
journal = {Magnetic Resonance in Medicine},
volume = {84},
pages = {2724– 2738},
abstract = {Purpose This study investigates the implications of all degrees of freedom of within-scan patient head motion on patient safety. Methods Electromagnetic simulations were performed by displacing and/or rotating a virtual body model inside an 8-channel transmit array to simulate 6 degrees of freedom of motion. Rotations of up to 20° and displacements of up to 20 mm including off-axis axial/coronal translations were investigated, yielding 104 head positions. Quadrature excitation, RF shimming, and multi-spoke parallel-transmit excitation pulses were designed for axial slice-selection at 7T, for seven slices across the head. Variation of whole-head specific absorption rate (SAR) and 10-g averaged local SAR of the designed pulses, as well as the change in the maximum eigenvalue (worst-case pulse) were investigated by comparing off-center positions to the central position. Results In their respective worst-cases, patient motion increased the eigenvalue-based local SAR by 42%, whole-head SAR by 60%, and the 10-g averaged local SAR by 210%. Local SAR was observed to be more sensitive to displacements along right?left and anterior?posterior directions than displacement in the superior?inferior direction and rotation. Conclusion This is the first study to investigate the effect of all 6 degrees of freedom of motion on safety of practical pulses. Although the results agree with the literature for overlapping cases, the results demonstrate higher increases (up to 3.1-fold) in local SAR for off-axis displacement in the axial plane, which had received less attention in the literature. This increase in local SAR could potentially affect the local SAR compliance of subjects, unless realistic within-scan patient motion is taken into account during pulse design.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Purpose This study investigates the implications of all degrees of freedom of within-scan patient head motion on patient safety. Methods Electromagnetic simulations were performed by displacing and/or rotating a virtual body model inside an 8-channel transmit array to simulate 6 degrees of freedom of motion. Rotations of up to 20° and displacements of up to 20 mm including off-axis axial/coronal translations were investigated, yielding 104 head positions. Quadrature excitation, RF shimming, and multi-spoke parallel-transmit excitation pulses were designed for axial slice-selection at 7T, for seven slices across the head. Variation of whole-head specific absorption rate (SAR) and 10-g averaged local SAR of the designed pulses, as well as the change in the maximum eigenvalue (worst-case pulse) were investigated by comparing off-center positions to the central position. Results In their respective worst-cases, patient motion increased the eigenvalue-based local SAR by 42%, whole-head SAR by 60%, and the 10-g averaged local SAR by 210%. Local SAR was observed to be more sensitive to displacements along right?left and anterior?posterior directions than displacement in the superior?inferior direction and rotation. Conclusion This is the first study to investigate the effect of all 6 degrees of freedom of motion on safety of practical pulses. Although the results agree with the literature for overlapping cases, the results demonstrate higher increases (up to 3.1-fold) in local SAR for off-axis displacement in the axial plane, which had received less attention in the literature. This increase in local SAR could potentially affect the local SAR compliance of subjects, unless realistic within-scan patient motion is taken into account during pulse design.