Three-dimensional (3-D) reconstruction of the spine surface is of strong clinical relevance for the diagnosis and prognosis of spine disorders and intra-operative image guidance. In this paper, we report a new technique to reconstruct lumbar spine surfaces in 3-D from non-invasive ultrasound (US) images acquired in free-hand mode. US images randomly sampled from in vivo scans of 9 rabbits were used to train a U-net convolutional neural network (CNN). More specifically, a late fusion (LF)-based U-net trained jointly on B-mode and shadow-enhanced B-mode images was generated by fusing two individual U-nets and expanding the set of trainable parameters to around twice the capacity of a basic U-net. This U-net was then applied to predict spine surface labels in in vivo images obtained from another rabbit, which were then used for 3-D spine surface reconstruction. The underlying pose of the transducer during the scan was estimated by registering stacks of US images to a geometrical model derived from corresponding CT data and used to align detected surface points. Final performance of the reconstruction method was assessed by computing the mean absolute error (MAE) between pairs of spine surface points detected from US and CT and by counting the total number of surface points detected from US. Comparison was made between the LF-based U-net and a previously developed phase symmetry (PS)-based method. Using the LF-based U-net, the averaged number of US surface points across the lumbar region increased by 21.61% and MAE reduced by 26.28% relative to the PS-based method. The overall MAE (in mm) was 0.24±0.29. Based on these results, we conclude that: 1) the proposed U-net can detect the spine posterior arch with low MAE and large number of US surface points and 2) the newly proposed reconstruction framework may complement and, under certain circumstances, be used without the aid of an external tracking system in intra-operative spine applications.