Tag: graphics processing units

TensorRT is a library developed by NVIDIA for faster inference on NVIDIA graphics processing units (GPUs). It can improve inference time for many real-time services and embedded applications, 4-5 folds. The optimization TRT applies to deep learning models will be examined in this article.