Buch, Englisch, Band 66, 236 Seiten, Format (B × H): 170 mm x 240 mm, Gewicht: 354 g
Buch, Englisch, Band 66, 236 Seiten, Format (B × H): 170 mm x 240 mm, Gewicht: 354 g
Reihe: Aachener Informatik Berichte Software Engineering
ISBN: 978-3-8191-0715-3
Verlag: Shaker
Deep Learning (DL) has advanced rapidly with larger models and datasets, driven by the belief that scale improves performance. However, this overlooks energy efficiency and environmental impact. This work introduces data- and knowledge-efficient training to address scaling inefficiencies in Computer Vision.
We propose a data-efficient SGD variant that biases optimization toward important samples identified after few epochs—no extra overhead unlike SOTA methods. We apply it to super-resolution of low-resolution CT scans (reducing patient radiation/costs), enabling accurate nasal cavity flow simulations.
For knowledge transfer, we develop an alternating training scheme using auxiliary vision datasets to boost the main MDE task via weighted steps, improving performance without full retraining.
Finally, ILoRA (Feature-Integral LoRA) offers compute-/memory-efficient fine-tuning: it uses fixed feature integrals for compression and a single trainable vector for decompression—fewer parameters per layer than SOTA PEFT.
Results show faster training with no performance loss by reducing data reliance, plus gains from auxiliary tasks and foundation model transfer. Combined with hardware advances, these strategies mitigate DL scaling's energy costs for sustainable high-performance AI.
