Light-scattering spectroscopy (LSS) is an established optical approach for nondestructive characterization of biological tissue. Here, we investigated the capabilities of LSS and convolutional neural networks (CNNs) to quantitatively characterize the composition and arrangement of cardiac tissues. We assembled tissue constructs from 200 µm thick sections of fixed right ventricular myocardium and aortic wall. In the assembled constructs, the aortic sections represented fibrotic tissue and the depth and volume fraction of these fibrotic insets were varied. We gathered spectra from the constructs at multiple locations relative to a light source. We used single and combinations of two spectra for training of CNNs. With independently measured spectra, we assessed the accuracy of the trained CNNs for classification of tissue constructs from single spectra and combined spectra. Classification accuracy with single spectra was smaller than with combined spectra. Overall classification accuracy was highest (mean ± standard deviation: 94.17±8.0%) for combined spectra from collection fibers most proximal to and distal from the illumination. Our studies demonstrate the reliability of quantitative characterization of tissue composition and arrangements using a combination of LSS and CNNs. Potential clinical applications of the developed approach include intraoperative quantification of atrial fibrosis and identification of conduction tissue in the heart.